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ABSTRACT 



This report on Phase III, of a four-phase study, provides an 
analysis of the impact of California Technology Assistance Programs. The 
first section of this report provides the background of the California 
Educational Technology Assessment Project (CETAP) , an overview of programs 
studied, and a summary of the evaluation plan. Sections two and three provide 
an analysis of the findings from the programs and projects studied in Phases 
I and II of the study. The analysis provides: (1) a brief summary of the 

program evaluated; (2) the evaluation questions and responses; (3) findings 
from Phase I and II supporting the responses; and (4) recommendations for the 
program. The final section of the report provides general conclusions and 
recommendations across all programs studied. An appendix contains the 
cost -benefit analysis of the six programs conducted by American Institutes of 
Research. ( AEF) 
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I. Background and Overview 



Report Overview: The first section of this report provides the background of the California 
Educational Technology Assessment Project (CETAP), an overview of programs studied, and a 
summary of the evaluation plan. Sections Two and Three provide an analysis of the findings 
from the programs and projects studied in Phases I and II of the study. The analysis provides: 1) 
a brief summary of the program evaluated, 2) the evaluation questions and responses, 3) findings 
from Phase I and II supporting the responses, and 4) recommendations for the program. The 
final section of the report provides general conclusions and recommendations across all programs 
studied. Appendix A provides the cost-benefit analysis of the six programs that was conducted 
by American Institutes of Research. 

A. Background 

Assembly Bill 1470, the Farr-Morgan-Quackenbush Educational Technology Act of 1989 
requires the California Department of Education (CDE) to provide support for an independent 
evaluation of educational technology programs in the state to “include procedures to standardize 
evaluation data collection, and strategies for the effective utilization of evaluation information to 
develop policy for use of educational technology.” (Chapter 1334, Section 51876.5[d], Statutes 
of 1989) The Far West Laboratory for Educational Research and Development (FWL), 
subcontracting with American Institutes for Research (AIR), was selected to conduct the 
evaluation. 

FWL was responsible primarily for the following: 1) evaluation of the 19 AB 803 programs and 
projects; 2) evaluation of five AB 1470 programs, including the School-Based Educational 
Technology Grant program, the Level II Academic Model Technology School Projects, 
Instructional Television (ITV), the California Technology Project (CTP), and Software 
Development; and 3) production of evaluation templates and guidelines for 1 and 2 above. AIR 
was responsible primarily for the evaluation of the six Level I Model Technology Schools (MTS) 
Projects and a cost-benefit analysis of the six programs included in Phase II of the study. The 
CETAP was conducted between October 1990 and July 1991. This analysis is the third of a five 
phase study as outlined below: 



Phase l A descriptive study of selected educational technology projects and programs funded by AB 
803 from 1984 to 1989. 

Phase II A descriptive analysis and a review of the findings of an in-depth formative and summative 
assessment of the impact of six major programs currently funded by AB 1470. These six 
programs are: 1) the first cycle of schools receiving School-Based Educational Technology 
Grants; 2) Level I Model Technology Schools; 3) (Academic-Technology) Level II Model 
Schools Projects; 4) Instructional Television Regional Agencies,; 5) Software Development 
Projects; and 6) the California Technology Project and its regional consortia. 

Phase III An analysis of the information collected in Phases I and II that provides program and policy 
recommendations to the stakeholders of the programs and projects studied. 

Phase IV Evaluation templates and guidelines to guide project and program staff in gathering 

information to provide standardized evaluation reports on future programs funded by AB 
1470. This package includes: 1) program evaluation guidelines and instructions, 2) program- 
specific self-assessment inventories, 3) formatted templates to be used to organize and 
develop program evaluation reports, and 4) surveys and other resources to support data 
gathering and reporting. The guidelines and templates will be used and revised as needed for 
future periodic evaluation of programs and projects. 

Phase V An executive summary of all phases of the study will be developed and distributed to 
educators and decision makers. 
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B. Programs Studied in Phases I and II 

AB 803 Local Assistance Programs. Phase I of the CETAP provided a descriptive analysis of 
14 programs and projects, selected by the CDE for the study, funded by the Educational 
Technology Local Assistance Program, Assembly Bill 803 (Chapter 1 133, Statutes of 1983), 
between 1984 and 1989. 

Assembly Bill 803 greatly expanded previous educational technology legislation. In fiscal year 
1984-85, AB 803 programs received $15 million in legislative appropriations. Funding for 1985 
and 1986 grew to $26 million, but in 1987 the budget was cut back to $13 million. During the 
life of AB 803, the Legislature provided over $60 million for school-based Adoption/Expansion 
Grants awarded competitively to schools across the state. AB 803 continued the state-wide 
network of seven regional ITV agencies, and more recently, initiated the California Technology 
Project (with resource consortia in 14 regions). AB 803 also funded the Technology in 
Curriculum (TIC) projects, Computer Software and Instructional Video Clearinghouses, 

Software Development Projects, VCR distribution, Summer Technology Training Institutes, the 
Model Technology Schools projects (MTS Level I), Developmental and Dissemination Projects, 
a Teaching Videotape Pilot Project, the Academic Model Technology Projects (MTS Level II), 
and several additional projects. 

In August of 1988, the Office of the Legislative Analyst issued a report, Educational Technology 
Local Assistance Program: Sunset Review, which concluded that the value and educational 
benefits of the array of AB 803 programs had not been sufficiently evaluated. This was used by 
the Governor as one of the reasons to cut the AB 803 funding by 50% and let the legislation 
sunset. 

AB 1470, Farr-Morgan-Quackenbush Educational Technology Act of 1989. Phase II of 
this study provided an indepth study of the programs funded by AB 1470 (Chapter 1334, Statutes 
of 1989). AB 1470 continues elements of AB 803, including a significantly modified version of 
Adoption/Expansion Grants, now known as School-Based Educational Technology Grants. AB 
1470 added a new program of Research and Development Grants and continues funding for the 
six MTS Level I projects and six MTS Level II sites. The act includes another new program 
category, the “Regional Assistance Program,” that re-authorizes the California Technology 
Project (CTP) and the regional ITV Agencies. 

State Program Evaluation and Planning. One of the conditions for funding both the re- 
authorization of the existing educational technology programs and the establishment of new 
programs under AB 1470 was the "independent evaluation of all program components.” This 
provided the mandate for the CETAP. 

This legislation reestablished the Educational Technology Committee and increased 
representation from K-12 teachers, higher education faculty members, and business leaders. It 
also established the California Planning Commission for Educational Technology to develop a 
long-range educational technology master plan for the state. The master plan will be based on 
outcomes from this study, input from the Educational Technology Summit, the Educational 
Technology Committee, the CDE, and leaders in business and industry. 

Emphases of AB 1470. AB 1470 also defines major areas of emphasis for programs prescribed 
in the bill including school-based technology use planning, coordination with state curriculum 
and instruction initiatives, effective staff development, and evaluation of the impact of the 
programs on teaching and learning. These areas of emphasis guided the final analysis of the 
findings of this study. 
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Governance. The Educational Technology Committee reviews budget and program 
recommendations made by the CDE and other stakeholders and makes final recommendations to 
the State Board of Education (SBE). The SBE makes final program implementation and funding 
decisions. The CDE is responsible for the day to day administration of the programs funded 
under the provisions of AB 1470. The CDE Office of Educational Technology provides for: 1) 
development of guidelines and program application documents, 2) training, 3) technical 
assistance, 4) overall budget management, 5) program evaluation and monitoring, 6) ensuring 
alignment of programs with state initiatives, and 7) contract and grants management and other 
duties as needed. 

AB 1470 Statewide Program Implementation. Since 1989, the CDE has cumulatively 
allocated approximately $28 million to programs authorized by AB 1470. From 1989-1991, 
programs included: 1) School-Based Educational Technology Grants, 2) Level I Model 
Technology Schools, 3) Level II Model Technology Schools, 4) the California Technology 
Project, 5) Instructional Television regional agencies, and 6) other smaller contracts and 
projects, including video program acquisitions, the CETAP evaluation contract, Video and 
Software Clearinghouses, and consultant services. The distribution of funding for 1989-90 and 
1990-91 for these programs and projects is shown in Figure 1. 

Figure 1: Educational Technology Expenditures for 1989-90 and 1990-91 
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The projected budget for 1991-92 adds two new programs: (1) the Research and Development 
program which will competitively fund two projects designed to develop, validate, and 
disseminate technology applications for Limited English Proficient (LEP) students, and (2) The 
Ancient World, an instructional materials development project to produce technology-based 
materials that support the California History-Social Science curriculum framework. The 
projected distribution of funding for 1991-92 is shown in Figure 2. 
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Analyses of Programs 



Figure 2: Educational Technology Expenditures for 1991-92 
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Overview of State Funded Educational Technology Programs. When combining AB 803 and 
AB 1470, the State has cumulatively funded over 30 educational technology programs and 
projects since 1984. The combined budget for all of these programs is approximately 
$102,000,000. The following pages provide the major findings from the intensive study of these 
programs and projects. Figure 3 shows all programs evaluated in both Phases I and II and the 
approximate timeframe for their development and implementation. 



Figure 3: Major Educational Technology Programs Funded from 1984-91 
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C. The Evaluation Plan 

Addressing Legislative and CDE Priorities. The evaluation design emphasizes both formative 
assessment of program implementation and summative assessment of educational and program 
outcomes. Every effort was made to conduct the evaluation with consistency across projects and 
programs. The evaluation reports can be expected to serve as primary sources of information for 
the California Planning Commission for Educational Technology and the CDE in developing the 
state master plan for educational technology. In general, the evaluation study should be a key 
element in shaping future policy, and in the planning, implementation, and evaluation of 
educational technology in California. 

Major Emphasis. This study was designed to assess the impact of the programs in terms of 
implementation of program priorities established by the Educational Technology Committee, the 
CDE, and AB 1470. In the evaluation plan, these priorities were categorized into a set of six 
areas of evaluation emphasis. The six areas of emphasis include: 



1. Site-Based Technology Use Planning. Both AB 1470 and the CDE stress the need to integrate 
planning for the use of technology into the existing school-level educational planning process. The 
degree to which educational technology is incorporated as an instructional strategy into school-level 
planning is assessed. 

2. Level of Implementation. CDE suggests that the level of implementation of a program affects the 
success of the program. The evaluation assesses the level of implementation of all programs 
including examination of the constraints and enhancement factors that affected implementation of 
the programs. 

3. Curriculum Support and Alignment. Critical to the CDE priorities is the degree to which the 
educational technology programs stimulate classroom applications of the State Curriculum 
Frameworks. To the extent possible, contributions to curriculum alignment are assessed for all of 
the programs included in Phase II. 

4 . Staff Development. The study documents and attempt to assess both the level of impact and the 
delivery systems for staff development made possible by the Educational Technology Local and 
Regional Assistance Programs. 

5. Learning Resources Management. The CDE has suggested that technology and other learning 
resources have not been optimally utilized within many instructional programs. This evaluation 
documents the extent of learning resources management by programs and projects. 

6. Evaluation and Accountability. AB 1470, the Legislative Analyst, and the CDE agree that past 
educational technology programs have lacked sufficient evaluation to determine their effectiveness. 
The evaluation project assesses both formative and summative evaluation efforts made by the six 
programs. 



D. Analysis of the Impact of the Educational Technology 
Programs 

Analysis of Programs authorized from 1989-1992: Phase II provided a comprehensive 
assessment of six programs and projects funded by AB 1470. Section II of this report addresses 
the evaluation questions included in the RFP and others added by the researchers. Data from the 
Phase II of CETAP is provided to support the conclusions. Recommendations for program 
changes and improvements are also provided. Listed below are the programs and projects 
studied in Phase II. 
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1. School-Based Educational Technology Projects (300 sites) 

2. Level I Model Technology Schools (6 projects) 

3. Level II Model Technology Schools (6 projects) 

4. Instructional Television Agencies (7 Agencies) 

5. California Technology Project (Central Agency and 14 consortia) 

6. Software Development Projects (4 projects) 



Analysis of Programs funded from 1984-1989. Phase I provided descriptions of 14 
components of the Educational Technology Local Assistance Program that were funded between 
1984 and 1989 by AB 803 described in Section III of this document. These include: 



1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 
9. 

10 . 

11 . 

12 . 

13. 

14. 



Adoption/Expansion Projects 
VCR Distribution 
ITV License/Program Acquisition 
TIC Guides 
TIC Software 

Summer Technology Training Institutes 
California Video Clearinghouse 
California Software Clearinghouse 
Teacher Training Video Project 
California Historical Society 
Developmental Grants 
Dissemination Program Grants 
ETN Staff Development 
Mechanical Universe 



The descriptions and findings related to these programs have been discussed in detail in Phase I 
of the study. The analysis and the final report will address the following as appropriate in 
relation to programs studied under Phase I of the study: 



1. What are the major strengths and weaknesses of each program studied in 
Phase I? 

2. What are the recommendations about particular practices and programs that 
should be reauthorized, continued, expanded or reduced? 

Note: Seventeen regional TEC Centers were augmented with AB 803 funding to support 
implementation, staff development, and evaluation related to AB 803 programs and projects. 
Reference is made to the TEC Centers in the descriptions and analysis on the following pages. 
TEC Centers were not targeted as part of this study. 

Overall Analysis, Conclusions and Recommendations. Section IV of this report reviews the 
overall effects of the programs on the state’s initiatives and priorities. Included in this section is 
the cost benefit analysis conducted by AIR. AIR reviewed the data collected to determine 
indicators of the relative cost benefits of the programs studied in Phase II. 
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II. Analysis of Educational Technology Programs 
Funded From 1989-1991 



This section addresses the Phase II questions (listed in Section I, Figure 4 of this report) that 
were derived both from the legislation and the CDE priorities. The format of this section lists the 
questions in italics, with the general responses in bold typeface, and support statements from the 
study listed with bullets. For each program studied, the questions and answers are generally 
related to background, planning, content, implementation, resources, funding, outcomes, and 
recommendations. Several other questions were added to those required by the RFP. For details 
about each of the projects studied, see Volumes I through VI of the CETAP Phase II Report. 
After most statements of findings, the data source appears in parenthesis. These data sources are 
found in Phase II, Volume I, the CETAP report. 

A. School-Based Educational Technology Grants 

1. Background Information 

Description: School-Based Educational Technology Grants are awarded to schools on a 
competitive basis for one year. The grants have a base of $2,000 per school plus $25 per 
participating student. Grants are awarded to plan and implement school-based educational 
technology projects to: 1) develop new technology-based programs, 2) expand existing 
technology-based programs, and/or 3) adopt or adapt existing validated programs such as those 
developed by the Level I and II Model Technology Schools. School-Based projects should be 
coordinated with site-based planning, increase the level of use of technology, facilitate the 
integration of technology into the curriculum, promote effective management of learning 
resources, include staff development, and evaluate the impact of technology on teaching, 
learning, planning, and resource utilization. 

Status: At present, approximately 600 school-based educational technology projects have been 
awarded serving about 10% of the schools in California. Over 1700 schools completed 
technology use plans in the process of applying for grants. During 1991-92, grants will be 
awarded to fund an additional 250 projects. However, this study focused on the 221 grants 
funded in the Spring of 1990 to be implemented during the 1990-91 school year. 

Data Sources: The key to the major data sources used for the assessment of the School-Based 
Grant projects is as follows: 

1 = Self-Assessment Inventory completed by the project directors 

2 = teacher survey 

3 = student survey 

4 = information from site visits and telephone interviews, 

5 = observations and judgement of CETAP staff. 



2. Planning 

To what extent were there changes in site-based planning for technology? 

The grant application process and guidelines for proposal development greatly 
facilitated inclusion of technology into local instructional planning and school-site plans. 
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• In a comparison between grant and non-grant sites, teachers at the grant sites 
(2) > ° rted that thCy had mUCh S 163161 " involvement in planning for technology use. 

• As recommended in the guidelines, project planning was often integrated with 
the activities of existing groups, such as School Site Councils and Chapter I 
Committees. Existing school and district Technology Use Plans and/or School 
Improvement Plans incorporating technology use were consulted by almost all of 
the projects as they implemented programs. (1) 

• Every project (100%) reported that technology use had been incorporated into its 
site planning. (4) 

• Almost all of the projects (90%) indicated that significant changes had been 
made in one or more areas from what was originally planned. (1) 



Was planning assistance provided by the California Technology Project and the CDE? 

Regional agencies such as the California Technology Project (CTP) with much 
assistance from the CDE facilitated technology use planning at the project sites. 

• The use of regional services, such as the ITV agencies and the California 
Technology Project Consortia, for planning assistance was greatly encouraged 
by the application process. Over half of the School-Based project directors had 
attended a CTP sponsored workshop, and one quarter used ITV agency services. 
The limited use of ITV agency services was due to the fact that most projects 
emphasized computer technology. (1) 

• The CDE developed the guidelines, training materials, and provided initial 
training of the CTP and ITV staff on how to develop and implement the 
Technology Use Plan. 

• Many teachers complimented the services provided by CTP and partially 
credited the regional consortia with their successful applications for grants. (4) 

• The CTP Consortia conducted a total of over 100 workshops and provided a 
great deal of follow up to assist School-Based Grant developers. 



3. Project Content 

To what extent did the School-Based Grants support the state curriculum frameworks? 

The planning, development, and implementation of School-Based Grants greatly 
facilitated the infusion of technology into the curriculum-especially English-language 
arts. 

• The most common primary grades curriculum emphasis for projects was 
English-language arts (72% of projects). This is because most projects utilize 
computers for writing. The most common secondary curriculum emphasis was 
History- Social Science. (1) 

• The subject area emphasis of projects tends to follow with the California 
Curriculum Framework implementation cycle. (1) 

• Integration of technology with the curriculum was a major area of emphasis for almost 
all of the projects (92%). (1) 




8 



12 



Programs Funded from 1 989-9 1 
School-Based Educational Technology Grants 



To what extent did School-Based Projects adopt or adapt Academic Model Technology 
Schools Projects (Level II MTS) ? 

The Adoption or adaptation of “model” programs facilitated implementation of 
School-Based Projects by providing successful strategies that had been tested 
and refined. 

• Close to one-third of the projects adopted or adapted a Level II Model Technology 
School project. 

• There was a tendency to adopt projects that focused on English-language arts. 

• There was limited number of adoptions of Level I MTS projects because they were just 
beginning to disseminate their programs and practices at the time of this study. (1) 



What types of technology were typically incorporated into the School-Based Projects? 

Most projects emphasized computers with a combination of other technologies. 

• Most projects (92%) placed a major emphasis on the use of computers. (1) 

• Eleven percent stated that use of Instructional Television (ITV) received major 
emphasis, and 24 percent stated that ITV received minor emphasis. (1) 

• Thirty-four percent of the projects listed laserdisc use as a major area of emphasis, and 
31 percent listed audio/video production. (1) 



What types of software and video were typically incorporated into the School-Based Projects? 

Projects tended to use application software with word processing as the major emphasis 
with minimal use of educational content-specific software. 

• As would be expected from the English-language arts emphasis of most of the projects, 
word processing and desktop publishing software received the greatest emphasis. (1) 

• Graphics was the second most used type of software, followed by multimedia. (1) 

• Grant recipients typically purchased programs such as Children s Writing and 
Publishing Workshop (a word processing and desktop publishing program), Apple 
Works (a word processing, spreadsheet, and database program), National Geographic's 
GTV (a laserdisc-based history and geography program). Software was most often 
purchased for use with Apple computers. (1) 



4. Project Implementation 

Were major activities implemented as planned? 

In general, the AB 1470 School-Based Grants Program was implemented as planned. 

To date, 595 projects have been funded and approximately 250 are to be funded in the 

Spring of 1992. 

• Project guidelines, funding formulas, distribution of RFPs, and training were provided, 
and districts were awarded funding according to the schedule established by the CDE and 
the Educational Technology Committee and approved by the State Board of Education. 

• Most projects were able to complete about half of what was expected by the end of the 
first year, and most activities were at least initiated. About half of the projects’ activities 
will be continued or repeated in 1991-92. (1) 
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• In general, project planners tended to be overly ambitious and to set unrealistic timelines 
for project implementation. (4) 



To what extent was project implementation coordinated with other programs? 

In general, efforts were made to implement the projects in coordination with other 
school programs. 

• In general, the projects were coordinated with other programs such as SIP, Chapter I, 
Chapter II, ESL, and others. (1) 



Was staff development available and did it meet needs of staff? 

Staff development was reported to be a critical component for successful projects. 

• Staff development workshops were approximately divided into thirds between 
awareness level, in-depth training, and in-depth with follow up assistance to the 
site. (1) 

• Projects generally allocated approximately 10 percent of their funding for staff 
development. 

• Most workshops (88%) were targeted for teachers, with a third for administrators; 
approximately 25% included classified staff. (1) 

• In general, the staff developers were highly rated by the project directors; most of 
the activities (81%) would be recommended to other educators. Most (80%) of 
the directors reported that the activities met the needs of the project. (1) 

• Staff development was viewed as critical to the success of the projects by many of 
the project leaders. (4) 

• A colleague was most often cited as the desired source of staff development. (1) 



To what extent were efforts made to evaluate the projects? 

Projects developed evaluation plans which were still being implemented when this study 
was conducted. 

• Most of the project staff (90%) indicated the project plan included on-going 
assessment activities, and three-quarters stated these activities were implemented 
as planned. (1) 

• A majority of the project directors (70%) had attended a CTP-sponsored 
evaluation workshop, and about half used the Educator's Guide for Evaluating 
Educational Technology Programs which was distributed by the CTP. (1) 

• About half the projects indicated that the planning committee reviewed formative 
evaluation findings. (1) 

• The most commonly used evaluation data sources were: teacher surveys (87%), 
teacher assessment of student work (87%), classroom observations (84%), and 
incidental comments by students and staff (84%). (1) 

• As suggested by the provisions of AB 1470, several projects utilized the self- 
study (37%), Program Quality Review (24%), and WASC review (4%). These 
percentages would have been higher if such site reviews had been conducted 
during the funding cycle of the AB 1470 project. The schools whose reviews 
coincided with AB 1470 project implementation consistently reported positive 
results. (1) 
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• Many projects had not completed their evaluations due to delays in 
implementation. (1) 

• Forty-three percent of the projects indicated that they would complete the optional 
end-of-year narrative evaluation report. School Committees (67%), school staff 
(66%), and district administrators (47%) would be the major recipients of these 
reports. (1) 

• Many project leaders reported the grant requirement to evaluate was the main 
incentive to conduct an evaluation. (4) 



To what extent did the adaptation of MTS Level I or II practices effect implementation? 

Adaptation of Level I and II MTS projects provides models to facilitate 
implementation, and helps prevent reinventing practices already developed and studied. 

• Several tested programs, staff development opportunities, and instructional 
materials aligned with the state curriculum frameworks are available from Level I 
and Level II MTS projects. (5) 

• These projects provide opportunities for educators to visit classrooms that are 
successfully integrating technology with the curriculum. Visitors can talk to 
teachers, administrators, and students to find out what works and what doesn’t. 

(5) 

• Expert planning and technical assistance is provided, eliminating the need for 
School-Based Projects to learn by trial and error. (5) 

• MTS staff resources to assist School-Based Projects to replicate and implement 
MTS projects were very limited. (5) 



To what extent did the program implement the CDE goals and initiatives? 

Planning, management of learning resources, framework utilization, staff development, 
coordination with school programs, and technology integration were improved or 
increased at most of the projects. 

• Most project staff reported that the project had a moderate to very moderate effect 
on the management and coordination of learning resources. (1) 

• A moderate increase in staff familiarity with and use of the California Curriculum 
Frameworks was reported. (1) 

• Most (60%) stated that many aspects of the project were incorporated into school 
plans. (1) 

• Most project staff stated that project planning activities helped to implement the 
CDE goals and initiatives. School-based planning, integration of technology into 
the curriculum, and staff development were the state initiatives most emphasized 
by the projects. (1) 

5. Support Resources 

What support services were used by school-based projects? 

District office staff and the CTP were the most common external sources of assistance. 

• School district offices provided the majority of support services to the projects.(l) 

• About half of the projects indicated the district had appointed a staff person to 
assist with implementation. (1) 
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• About half of the schools used California Technology Project (CTP) regional 
consortia services. The services most used were: proposal preparation 
workshops, the Educator' s Guide for Evaluating Educational Technology 
Projects, and individualized assistance. (1) 

• With the exception of the CTP, many projects indicated a lack of awareness of 
available support resources such as ITV agencies and MTS projects. 

• Level II Model Technology Schools services, including printed instructional 
materials, implementation guidelines, individual assistance and workshops were 
used by about one quarter of the projects. (1) 

• A limited number of projects made use of the services of the Instructional 
Television Regional Agencies or Level I Model Technology Schools. (1) 

• Most of the project directors (87%) were aware of the CTP and the services that it 
provides. (1) 

• The CTP resources most often used were AB 1470 Grant Preparation Workshops, 
technical assistance in person or by telephone, TRIE, and individualized 
assistance. (1) 

• School districts and contract consultants were the most common sources of both staff 
development and technical assistance. (1) 

• The types of technical assistance received most often were installation and operation of 
equipment and problem solving/trouble shooting. (1) 



To what extent were evaluation resources used? 

Other than district and school staff, the CTP and the CETAP-produced Self-Assessment 
Inventory were the major sources of evaluation assistance to School-Based Grant 
projects. 

• The most commonly used sources of evaluation assistance were teachers, school 
principals, district office staff, CTP evaluation workshops and staff, and the 
Educator's Guide for Evaluating Educational Technology Programs. (1) 

• Over half of the respondents reported that the Self-Assessment Inventory served 
as a working document to guide evaluation, but many stated that it would have 
been more useful if it had been received at the beginning of the year. (1) 



6. Funding Support 

To what extent were technology applications continued after project funding ended? 

Almost all projects stated that at least some aspects of the project would continue after 
the AB 1470 funding was expended. 

• Over half of the projects will continue and expand at the school site after the 
termination of AB 1470 funding. (1) 

• Several projects stated that they will continue to purchase software and conduct 
staff development. (1) 



To what extent were the project related activities funded by sources other than AB 1470? 
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Significant additional funding was received from districts and a variety of other 
sources. 

• Most projects actually contributed more than the required ten percent in matching 
funds. (1) 

• The most common sources of non-AB 1470 funding were district general funds, 
school general funds, the School Improvement Program, business/industry, and 
the Lottery. (1) 



To what extent was funding adequate for project implementation? 

AB 1470 funding was needed to get projects started, but in many cases it was not 
adequate or spread over enough time to fully fund implementation. 

• About half of the projects felt that the funding received from AB 1470 was 
adequate. (1) 

• Over half of the respondents stated that their projects would not have been 
implemented without the existence of AB 1470 funding. (1) 

• Many teachers suggested that projects should either be funded for a longer period 
or be eligible to apply for additional grants. (4) 



7. Supporting and Impeding Factors 

What were the factors that most supported project implementation? 

Implementation was facilitated by motivation and commitment of students, staff 
combined with technology use planning, and access to technology-based resources. 

• Student and staff motivation and interest greatly facilitated project 
implementation. (1) 

• School and district administrative support greatly supported implementation. (1) 

• The proposal preparation process and technology use planning facilitated overall 
project implementation. (1) 

• Access to computer hardware and software was an important factor. (1) 



What were the factors that tended to impede project implementation? 

Impediments to implementation were time constraints and changes in student student 
and staff participants. 

• Many of the projects reported that implementation was behind schedule (61%) for 
reasons which include equipment procurement delays and activities that were too 
ambitious. (1) 

• Staff often indicated that the busy school day did not allow enough time for project 
planning and implementation. (4) 

• Most indicated that one year was not enough time to implement and that projects should 
span over two years. 

• Student and staff attrition were major impediments for project success. (1) 

• Class size was often reported as a major impeding factor in that it did not allow 
teacher time to plan and implement project-related classroom activities. (1) 



0 



Analyses of Programs 



8. Project Outcomes 



To what extent were the program outcomes attained? 

Most projects attained all of their student and teacher objectives. 

• Over two-thirds of the projects reported that they met all of their stated student 
objectives for the project. (1) 

• Almost all (94%) of the projects reported that they met all of their teacher 
objectives for the project. (1) 



To what extent did student performance ( academic and behavioral) improve? 

There were a variety of significant improvements in student performance and 
motivation. 

• There were significant increases of student interest and skill in using technology. 

• There were significant improvements in knowledge and skills for subject areas 
emphasized by the projects, quality of work completed and student initiative. (1) 

• Moderate to significant improvements were reported in problem solving and 

higher order thinking skills, interest in school, and classroom behavior and study 
skills. (1) 3 

• Most of the students believed their grades had improved as a result of using 
technology. (2) 

• Most students stated that computers improved their writing abilities. (2 - p. 45) 

• Many project leaders noted that student writing had been greatly enhanced. (4) 

• A majority of students stated that technology made school more interesting and 

enjoyable. (2) 6 

• In a comparison between AB 1470 and non-AB 1470 sites, an increase (34%) 
was seen in the teachers’ perceptions that report card grades of their class, as a 
whole, had improved as a result of technology use. 



To what extent were there desired changes in instructional practices and teacher performance? 

Teachers’ ability to use technology to support the curriculum was significantly 
improved. 

• There were significant improvements in support for student-centered learning, 
teacher interest level, ability to integrate technology with the curriculum, and 
confidence in ability to use technology. (1,3) 

• Overall, increased flexibility and greater variety in instructional methods was 
noted. (4) 

• Teachers were better able to expand the use of resources beyond the textbook, 
encourage problem solving and critical thinking, collaborate with other teachers, 
and provide interactive experiences with students. (1) 

• Teachers reported a moderate increase in use of the curriculum frameworks, more 
efficient and directed use of technology-based resources, and increased planning 
for student needs. (3) 
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• The project resulted in increased collegiality and collaboration. Many reported 
that teachers were working together and sharing more frequently. (4) 



To what extent were unanticipated outcomes assessed? 

Teachers reported many unexpected improvements in student learning. 

• Many at-risk students excelled at technology use and became leaders. (1,3) 

• Students learned to use technology more quickly than was expected. (1,3) 

• More training was needed than was anticipated. (1) 

• Technology was sometimes able to help students with physical and learning 
disabilities. (4) 

• Project implementation took longer than anticipated. (4) 



To what extent do teachers and administrators value the projects? 

Teachers and administrators place a high value on the improved access to technology 
resources provided by these projects. 

• Teachers placed a high value on the use of computers to support the instructional 
program, followed by instructional television and multimedia. (3) 

• Teachers surveyed reported that students and administrators value the use of 
technology in school more than parents and other teachers. (3) 



To what extent was there dissemination of knowledge and/or products? 

Though dissemination was not the focus of these projects, many developed promising 
practices and products that could have potential for use at other sites. 

• Over three-fourths of the projects reported they had produced instructional 
procedures, practices, or products that warrant consideration for continuation 
and/or dissemination to other classrooms or schools. (1) 

• Several directors stated that in the future their project would expand to other 
schools within or outside of the district. (1) 



To what extent was there equal access for students, teachers, and administrators? 

Allocation of grant resources for the first year of the program was equitable. 

• The selection of grants to be funded by both region and score resulted in an 
equitable distribution among schools in rural, urban, and suburban areas. (1) 

• Oyer half of the students served by the School-Based Grant projects were ethnic 
minorities. The distribution of students by ethnicity was almost the same ratio as 
for the whole state. (1) 

• The School-Based Grant projects benefited a great number of at-risk students in 
Chapter I schools, ESL and bilingual programs, alternative schools, and special 
education programs. Many teachers reported that the lowest achieving students 
benefited the most from the introduction of technology. (4) 

• Most respondents (80%) felt that students and teachers had equitable access to 
resources. (1) 
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9. Recommendations 

What aspects of the program should be continued, discontinued, expanded or modified? 

The School-Based Grants Program is successful and should be continued with resources 
added to provide follow-up and identification of promising practices. 

Planning 

• Continue the site-based planning requirement for the Technology Use Plan and 
encourage schools to refer to and follow their plan frequently. 

• In the guidelines and in the proposal development training, provide educators 
with information about establishing realistic project timelines. 

• Information about the regional support agencies needs to be made more available 
to schools. 

Funding 

• Continue to use the process for allocation of funding across the ten County State 
Steering Committee established regions. 

• Projects should be funded for a two year period and should be able to reapply for 
continuation funding if they are willing to disseminate promising products and 
practices. 

• Increase the level of funding for School-Based Grants Program and the support 
resources that clearly increased the effectiveness of the projects (e.g. CDE, CTP, 

ITV, and model programs). 

Staff Development 

• Continue to require that a portion of the budget be allocated to staff development. 

• Refine the application form to elicit a more complete description of the 
Technology Use Plan and how it will be developed and used. 

• Funding should be provided to support the recommendations made in this report. 

Evaluation 

• Evaluation forms should be made available to projects before implementation 
begins in order to guide data collection. 

• Develop and disseminate new “PQR Exemplars” that depict exemplary uses of 
technology within the instructional program to future project developers. 

• The CTP regions should play a major role in supporting evaluation activities and 
in collecting project-specific evaluation information required in the RFP. 

• The CDE should utilize a standard form for the project evaluation (e.g. an 
adaptation of the self-assessment inventory). 

• Applicants should be encouraged to involve the broader community, including 
local businesses, in planning the project. 

• The CDE should provide for year to year collection, analysis, and reporting of 
project data and information to stakeholders. 
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Dissemination 

• A dissemination system should be set up for the many projects that have 
developed promising products or practices. 

• The CTP should be used to disseminate promising products and practices. TRIE, 
workshops, TLAs, and other CTP services could be used. 

• A process with incentives should be established to identify and disseminate 
promising practices that emerge from the School-Based Educational Technology 
Grants program. 



B. Software Development Partnerships 

1 . Background Information 

Description: Between 1987 and 1991, seven software development partnership projects were 
funded for $1.9 million by the CDE to facilitate development of quality software and video 
programs to support all curriculum topics at all grades as appropriate. The state receives 
royalties on the out-of-state sales of these products. The TIC projects funded earlier by AB 803 
helped to establish the need for the software development projects. These matching grants were 
intended to provide seed money to encourage publishers to invest capital in return for guidance 
from the CDE. The CDE maintains the view that technology-based materials can provide 
students with the curriculum and interactive instruction that cannot be provided by textbooks and 
other print materials alone. The CDE also believes that improved technology-based materials 
(i.e., aligned with the curriculum frameworks) will encourage schools to purchase appropriate 
hardware. 

Status: Four of the seven projects ( Explorations in Science, Jostens Learning Corporation; GTV, 
National Geographic Society; Science 2000, Decision Development Corporation; and MECC 
World GeoGraph) have completed their contracts and the other three ( Exploring Matter, C&C 
Software; Adventures in Mathland, Mindscape; and Science and Society, Advanced Ideas) were 
cancelled before completion. These projects were all funded by AB 803. An eighth project, The 
Ancient World, will be funded in 1991-92 by AB 1470. 

Data Sources: The key to the major data sources used for the assessment of the software 
development projects is as follows: 1 = Self-Assessment Inventory completed by the developers, 
2 = user survey, 3 = CDE records, and 4 = comments and surveys from other projects. Data from 
these sources is described in detail in Volume VI of the CETAP Phase II report. 



2. Planning 

To what extent were the project development activities completed and objectives attained? 

Four of the projects were completed and three were cancelled. 

• The GTV project completed all of its activities and objectives with some changes 
to what was originally proposed. The major change was in curriculum emphasis, 
shifting from geography-only to a combination of history and geography. (1) 

• The MECC World GeoGraph and Jostens Explorations in Middle School Science 
projects completed their activities and objectives with no major changes. (1) 
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• The Science 2000 project completed all of its activities and major objectives, and 
is only recently being marketed. (1) 

• Three other projects, all funded at low levels, were cancelled before completion. 
(3) 



3. Content 

To what extent did the project support the California curriculum frameworks and other CDE 
initiatives? 

The programs developed in each of the successful projects were designed to support and 
expand the California curriculum frameworks. 

• Two of the completed projects targeted science, the other two targeted history- 
social science. Both of these framework areas were determined to be lacking in 
high-quality software. (1) 

• All of the successful projects made extensive use of the California curriculum 
frameworks. Model Curriculum Standards, and Technology in the Curriculum 
Resource Guides. (1) 

• GTV, Explorations in Science and Science 2000 provide extensive curriculum 
integration information. In addition. Science 2000 provides an entire year’s worth 
of seventh grade science lesson plans and support materials which the publisher 
considers to be comprehensive enough to replace conventional textbooks, 
although some textbook materials may be referenced and used. (1) 

• Over two-thirds of the respondents to the CETAP user survey decided to try 
World GeoGraph, Explorations in Science, or GTV with their classes because it 
related to curriculum needs of their students. (2) 

• With few exceptions, teachers reported that the curriculum areas targeted by the 
software products were covered adequately. (2) 



To what extent was staff development that supported implementation provided? 

AH of the projects but MECC World GeoGraph provide or will provide adopters with 
staff development. 

• Statewide, California schools received $300,000 worth of staff development in 
support of GTV. (1) 

• MECC reports that World GeoGraph is “user friendly” such that no training is 
required. ( 1 ) 

• Sixty hours of training are provided with Explorations in Science. (1) 

• Training and demonstration sites will be established for Science 2000. (1) 

• A school that purchases Science 2000 may elect to have a full day training session 
at their site. 

• All projects provided teacher-guidance materials. 
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4. Product Development 

To what extent was assistance provided by the CDE? 

The CDE provided considerable assistance in planning, developing, and marketing the 
software products. 

• The projects were all initiated by the CDE in collaboration with the software 
development partners who responded to requests for proposals. (1) 

• CDE involvement with project planning ranged from moderate to extensive. (1) 

• The CDE and contract consultants assisted in product development. Collaborative 
activities with the CDE included: meetings, evaluation of content and software 
quality, marketing to California schools, staff development planning and 
implementation, and monitoring of project progress. (1) 

• C & C was the only developer responding to the study that was dissatisfied with 
the level of support and collaboration obtained from the CDE. (1) 



5. Evaluation 

To what extent were the programs evaluated? 

All of the completed projects conducted beta testing and modified the products based 
on the results. 

• All of the software development projects tested the products in real-life classroom 
situations. (1) 

• Hundreds of teachers and thousands of students were consulted in the pilot testing 
process, using both surveys and classroom observations. (1) 

• Student and teacher suggestions were used in making changes and adjustments to 
the products before their final release. (1) 



6. Project Outcomes 

To what extent did student performance and motivation improve? 

Student ability to access and manipulate information in new and interesting ways was 
improved. Student motivation was reported to have increased. 

• Student outcomes reported by the developers of GTV included: traditional 
material presented in a vivid new form, access to an extensive database of images, 
a chance to practice reading and writing within the context of “making” television 
and videos specifically designed for a media-saturated audience. (1) 

• Students using World GeoGraph learned to use a database as a tool for inquiry 
and analysis, observe patterns of similarities and differences among nations, 
observe possible correlations among various types of data, and understand basic 
geographic concepts of location, place, relationships within places, movement, 
and regions. (1) 

• Explorations in Science helped students understand ideas outside of conventional 
experience and to easily observe the results of changing variables in physical 
science experiments. (1) 
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• Several teachers reported that GTV increased the motivation and problem solving 
ability of at-risk and ESL students. For example, one teacher described a shy, 
underachieving, ESL student who learned to use GTV by himself and with it 
produced a presentation that was “the best in the class.” (4) 

• Sixth grade students, when asked about World GeoGraph at one project site, 
replied: “It gives a choice. We can select what it is we want to explore.” and “It 
helps me in learning; I can find out for myself the information in the way I want 
to find it, without having to be programmed to do it in a certain way.” (4) 

• GTV was mentioned specifically by several students who were surveyed at 
School- Based Grant projects. These students found the program to be both useful 
and fun to work with. (4) 

To what extent did desired changes in instructional practices and teacher performance occur? 

The ability of teachers to integrate technology resources with the curriculum and 
encourage problem solving and critical thinking was improved. 

• GTV, Explorations in Science and World GeoGraph reported a significant 
increase in teacher abilities to expand resources beyond those available in 
textbooks and to make teaching more interesting. (1) 

• Other areas of strong positive impact on teachers were improvements in 
integrating technology with the curriculum and encouragement of problem 
solving and critical thinking. (1) 



To what extent do teachers value the programs? 

Teacher ratings of the educational value were high for all of the programs. 

• In general, the programs received high ratings for overall instructional value 
(average of 4.3 on a scale from one to five). 



To what extent were unanticipated outcomes assessed? 

The major unanticipated outcome was the cancellation of these projects. 

• Although the state acknowledges that software development projects involve 
some risk, only six percent ($1 14,000) of the nearly $1.9 million expended on 
software development went to unsuccessful projects. The financial instability and 
lack of curricular knowledge of the publishers not completing their contracts was 
difficult to predict. (3) 

• Three of the products have already returned $105,000 in royalties to the state on 
out-of-state sales. 



To what extent was there equal access to the programs for students, teachers, and 
administrators? 

Acquisition of the products by schools varied greatly due to cost and marketing efforts. 

• Two of the products, MECC World GeoGraph and GTV , are priced in a 
competitive range that most schools can easily afford. (1) 

• Site licenses are available for Science 2000 and Explorations on Science. 

• So far, only GTV and MECC World GeoGraph have been extensively promoted in 
California. Many demonstrations and free training sessions were conducted by 
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the National Geographic Society. World GeoGraph was promoted through 
advertisements in magazines, conference exhibitions, and the MECC catalog. 

Promotion for other products has just begun, therefore access to them is limited 
by a lack of awareness. (1) 

7. Funding and Support Factors 

To what extent was implementation supported or facilitated and why? 

Funding and support from the CDE facilitated project development and 

implementation. 

• The factors identified as providing the most support in implementing project activities 
were: the amount of funding provided by the CDE, overall funding for the projects, field 
testing of the software, and the results of the internal evaluation of the products. (1) 

• All of the projects reported that product development would not have been undertaken 
without support from the state. (1) 



To what extent were additional resources leveraged by the state funding? 

The CDE software development program leveraged considerably more money from the 
software publishers than had been invested by the state. 

• With the exception of Science 2000, all of the developers contributed funding in 
excess of the grant amounts provided by the state. (1) 

• The National Geographic Society and several partner organizations made the 
largest contributions in funding; the consortium expended $1.2 million in addition 
to the $500,000 contributed by the state in developing GTV. (1) 

• In all, over $2.3 million was invested by the software publishers compared with 
the investment of $1.7 million by the CDE (these figures do not include the three 
cancelled projects). (1) 



To what extent was implementation impeded and why? 

For the successful projects, there were few major impediments to project 
implementation. 

• According to the developers, the factor that most consistently impeded marketing 
was the limited hardware available at schools. However, one of the purposes of 
these projects was to provide an impetus for schools to purchase hardware. (1) 

• Reactions to CDE project monitors were mixed; two publishers reported that the 
monitor impeded progress and three reported that the monitor supported progress. 
( 1 ) 

• After seeing prototypes, the CDE often required substantial changes to the 
software products. Though normal for projects of this type, this was a major 
source of frustration to staff members at the cancelled projects. (1) 



8. Marketing 

To what extent was there dissemination of knowledge and/or products? 

Three of the products are currently being marketed both in California and nationwide. 
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• The World GeoGraph and GTV programs are marketed through catalogs, 
brochures, conference exhibits, advertisements, and other means. Both products 
have exceeded publishers’ sales expectations and have passed the break-even 
point on development costs: (1) 

• Explorations in Science is marketed directly by Jostens sales representatives for 
use in schools that have adopted the company’s integrated learning system which 
requires a network of microcomputers. (1) 

• Marketing efforts and outcomes have not been systematically or consistently 
monitored. (3) 

9. Recommendations 

What lessons have been learned that could improve future products? 

Policy lessons can be learned from both the successful and unsuccessful projects. 

• Future RFPs should indicate that preference in awarding contracts will be given to 
software publishers that are able to provide (1) staff development support for 
California schools that adopt the software project and/or (2) training-of-trainer 
staff development to staff members of appropriate regional support agencies, such 
as the regional consortia of the CTP and SB 1882, the Subject Matter Professional 
Development Projects, and other agencies. 

• The problems encountered in the management of the three cancelled software 
development partnerships has effected an improvement in setting standards and 
expectations by the CDE about the quality and content of the products to be 
developed. 

• A rigorous, uniform evaluation process, which includes guidelines and criteria for 
evaluating and revising prototype materials and focuses on performance-based 
assessment of student outcomes resulting from the software programs, should be 
included in future solicitations for software development projects. 

• Schools applying for School-Based Educational Technology Grants in the future 
should be made fully aware of the availability of state-developed software 
programs that support the curriculum frameworks 



To what extent should the state continue to fund software development projects? 

Future projects should shift some of the funding support from development to 
certification and dissemination of exemplary technology-based materials. 

• Careful consideration should be given to the availability of funding for 
technology-based materials in developing specifications for future software 
development projects supported by the CDE. 

• To encourage the use of interactive multi-media programs, the CDE should 
consider a state or group buy of laser disk players. All of the software 
development projects reported that the lack of equipment in the schools impeded 
their marketing efforts. 

• The CDE should establish procedures for tracking the software programs after 
production and distribution to assess their use and impact on teaching and 
learning. 

• The follow-up research should also assess the extent to which software 
development projects lead to increased teacher implementation of the California 
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curriculum frameworks and whether or not students develop and apply the 
knowledge in solving meaningful and relevant problems. 

• The CDE should continue to evaluate the impact and cost-benefits of existing 
software development projects such as Science 2000. 

• Pay close attention to and describe features of the current successful software 
development projects and ensure that new development projects (such as The 
Ancient World) consider such features. 

• The CDE should explore alternative incentives for publishers of technology-based 
materials to develop high-quality innovative products. 



C. Level I Model Technology Schools 

1. Background Information 

Description: California’s six Model Technology Schools Projects are funded to: demonstrate 
instructional technology use that supports state curriculum frameworks; develop quality teacher 
and administrator training models; support and disseminate research on the effective 
implementation of technology in schools; field test and promote the development of new 
information technology products; determine facility standards required for efficient use; and 
disseminate results. Expected outcomes of these projects include validated models for 
technology planning, implementation, and assessment that can be adopted or adapted by other 
schools. These five-year projects were first funded in 1987 by AB 803, with the last two years 
being funded by AB 1470. 

Status: The six districts currently funded are: Alhambra City Schools (two schools), Cupertino 
Union Elementary and Fremont Union High (three schools from two districts collaborating on 
one project), Los Angeles Unified (four schools), Monterey Peninsula Unified (four schools), 
Hueneme Elementary (one school), and Sacramento City Unified (three schools). These projects 
are in their final year of funding and are concentrating their efforts on packaging and 
disseminating promising products and practices. 

Data Sources: The key to the major data sources used for the assessment of the Level I Model 
Technology Schools projects is as follows: l=Self-Assessment Inventory, 2=Teacher Survey, 
3=Visitor Survey, 4=Student Questionnaire, 5=Site Visit Interviews. Data from these sources are 
described in detail in Volume II of the CETAP Phase II report. 

2. Planning 

Were major activities implemented as planned? 

AH projects reported a major emphasis and a large investment of resources on planning 
activities in the first few years of implementation. Some changes were reported in 
project orientation as they progressed through the years. 

• All sites reported an increased emphasis on dissemination than they had originally 
anticipated. (1) 

• Some sites also reported an increased emphasis on business partnerships and on moving 
to more decentralized project planning. (1) 
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Were there changes in site-based planning for technology? 

There were changes in site-based planning for technology as a result of these projects. 

• Over 50% of the respondents to the teacher survey report increased involvement 
in school decision making as a result of the increase in technology use. (2) 

• During the site visits, a number of respondents reported substantially increased broad- 
based interest in planning for technology use at the site. This varied by site, however, 
with some sites emphasizing bottom-up planning much more than others. (5) 



3. Content 

Did the projects support the state curriculum frameworks? 

All projects were attempting to design technology applications that supported the state 
curriculum frameworks. 

• Over 50% of the respondents to the teacher survey report improved alignment with the 
state curriculum frameworks as a result of the increase in technology use. (2) 

• Over 35% of the teachers responding to the teacher survey at the project sites reported 
that technology had been "very much" of use in supporting the curriculum frameworks. 
Another 42.3% said that technology had been of some use in this regard. (2) 

• Site level interviewees commented that the state might play a more active, central role in 
linking innovating technology applications to the guidelines in the state curriculum 
frameworks. (5) 



4. Project Implementation 

Was staff development that supported implementation provided? 

Staff training and development was reported across all six project sites. The amount of 
training varied considerably by site, however, and staff sometimes reported that 
training was insufficient and not always relevant to direct classroom applications. 

• The total number of training hours reported by the six projects for the 1989/90 and 
1990/91 years ranged from 155 to 5,500. There was little correlation between the size of 
the project and the number of training hours reported. (1) 

• There is a correspondence between the degree of training reported by project and the 
number of staff who reported that training was insufficient and not always relevant to 
direct classroom applications. (5) 

• Nearly 70% of the respondents to the teacher survey reported that the adequacy of the 
training they received had improved as a result of the increase in technology use. (2) 



Was project implementation coordinated with other programs? 

The degree of coordination with other programs varied considerably by project site. At 
most sites the program was well implemented into the whole school program and plant 
operations. 

• One of the most common areas of coordinated services was in the area of Limited English 
Proficient instruction. Many of the sites reported that they had successfully incorporated 
technology with this program. (5) 
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• Examples of the incorporation of technology with special education programming were 
also cited. (5) 

• At some sites, however, the cohort of trained technology teachers remained in isolation 
from the rest of the school. At some sites the equipment (e.g. the computer lab) was 
closed to teachers not in the cohort. (5) 

Was implementation restricted and why? 

Implementation was restricted in one way or another at nearly all of the project sites. 
The greater the prior commitment to and involvement with technology the less the 
degree of restriction. 

• Two of the six sites received notice of their grant award during the summer prior to the 
first year of implementation. This allowed them considerably greater time arid capacity 
to be ready to initiate implementation as school opened in the Fall. (5) 

• Most of the other sites were not notified of their awards until the Fall and did not receive 
equipment until Spring. This retarded implementation substantially. Some sites reported 
that it caused them to largely lose the first implementation year. (5) 

• Nearly all of the projects reported difficulties with equipment delivery. Some sites were 
in their second year of implementation before they really had their equipment on hand. 

( 5 ) 

• A number of projects cited the lack of clear direction from the CDE as a restriction to 
implementation. They reported that the state-wide goals they were attempting to achieve 
were unclear and/or underwent considerable revision. (5) 

Was implementation supported and facilitated and why? 

There were staff who worked very hard to support implementation at all six project 
sites. For some projects this reflected commitments from district, school, and project- 
management staff. The most successful projects received active assistance from all 
three of these important levels of support. Other projects only received support from 
one or two of these levels. 

• More than ninety-five percent of the teachers reported overall administration support as 
positive. (2) 

• At four of the six sites support from district, school, and project- management staff was 
clearly in evidence. The importance of technology use to instruction was identified as the 
major focus of the school and sometimes the district at these sites. At other sites, 
technology was simply one project among many. 

• The reason for the broad-based support, where it was received, appeared to be a genuine 
commitment on the part of the district to the use and development of educational 
technology. Districts showing evidence of a strong commitment to technology use and 
development in the district saw this project as the centerpiece, or an important 
component, of this effort. (5) 



Did the project implement the CDE goals and initiatives? 

All of the projects showed evidence of attempting to respond to CDE goals and 
initiatives in the implementation of the project. However, most of the projects also 
reported difficulties in this regard. 




25 29 



Analyses of Programs 



* Nearly all of the sites expressed some frustration with this, however, because they felt 
that these goals and objectives were not always clearly stated and communicated and 
sometimes represented a moving target. (5) 

• There was concern that the implementation of a project of this complexity and magnitude 
was difficult and a changing state agenda sometimes added to their difficulties. (5) 



To what extent have the MTS projects demonstrated the effectiveness of technology in 

improving the administration of school and district-level operations? 

The Level I projects appeared to be more geared to school rather than district-level 
technology applications for administrative uses. 

• The degree of emphasis on office administration varied across the six projects, which 
reported it as a major emphasis (three sites) to no emphasis (one site). Also, the percent 
of office staff who were considered to be active participants in the project varied across 
the projects from 17% to 100%. 

5. Support Resources 

Was assistance provided by the CTP? 

Some projects reported receiving assistance from the CTP. 

• Two of the six projects reported that little or no assistance was needed from the CTP. (1) 

• The other four projects reported that such assistance was needed, but only one of these 
four reported that they received the level of assistance that was required. (1) 

Was assistance provided by the CDE? 

Assistance was provided by the CDE, but some projects reported that it was not 
sufficient. 

• All of the projects reported that high level of assistance in the implementation of this 
project was needed from the CDE. (1) 

• Four of the six projects reported that the support received from the CDE was less than 
what was needed. The other two reported that it was sufficient to meet their needs. (1) 

6. Funding Support 

How were AB 1470 funds used in the projects? 

Funds were used quite differently across the projects. 

• Some projects emphasized personnel expenditures and allocated approximately 40% of 
project funds for the 1989/90 school year for salaries and benefits. (1) 

• Other projects emphasized capital outlay with two projects allocating over 40% of project 
funds for the 1989/90 school year in this category. (1) 

• Five of the six projects used contract staff to conduct the required evaluation component 
of the project, and all sites allocated between 10 and 25% of total project funds to 
evaluation services. (1) 



How did these funds compare to the total funding allocated to technology at project sites? 

AB 1470 funds did not constitute total revenues for technology expenditures at any of 
the projects for the 1989/90 year. 
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• AB 1470 funds were the revenue source for at least one-half of technology-related 
expenditures at all of the project schools. This ranged from 51.4% of total technology 
funds at the Sacramento project schools to 83.5% of all funds at the Alhambra project 
schools. (1) 

• Over 20% of the additional funds came from the district general fund for Los Angeles 
and Sacramento. (1) 

• Hueneme and Sacramento also used substantial amounts of other categorical funds for 
technology-related expenditures, at 25% and 17.6% respectively. (1) 

• Cupertino and Los Angeles reported the largest amounts from donations for the 1989/90 
year at 27.8% and 25%, respectively. (1) 



7. Outcomes 

Were the program outcomes attained? 

All sites reported that they had largely met the program goals set for themselves. 

• Numerous indicators of student and teacher satisfaction and other subjective outcome 
indicators were reported in conjunction with the project. 

• From a state project-wide perspective, however, the lack of uniform, project-wide 
evaluation criteria make this question difficult. The difficulties associated with this lack 
of clear, system-wide goals and evaluation criteria was noted by a number of the project 
directors and in a project-wide evaluation conducted for the CDE in 1989. 

Did the project improve the management of learning resources? 

The projects developed some important innovations in the management of technology 
learning resources. 

• All six project sites reported improved procedures for the management of learning 
resources. (1) 

• Some of these practical innovations, such as the Monterey MTS Level I classroom 
intervention plan, and the Cupertino MTS Level I personalized learning plans for 
teachers, are as important for dissemination purposes as the curricular packages, software 
applications, or other products developed through the project 

Did students performance improve? 

Although test results were mixed, all sites reported substantial gains in student 
performance. 

• Gains in such student outcomes as fewer class absences, completion of homework, and 
higher standards of performance were cited at all of the sites. (1) 

• All of the teachers at project schools were surveyed regarding the impact of the use of 
technology on student performance. The average positive response on ten separate 
criteria was 71.1%. (2) 

• A sample of students at project schools were surveyed regarding the impact of the use of 
technology on their academic performance. The average percentage responding 
positively on ten separate criteria was 74.6%. (4) 

• Hueneme reported gains in student test scores, but all of the other project sites expressed 
the belief that it is too soon to expect such changes or that this is an unrealistic goal. 
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Were there desired changes in instructional practices? 

Although the different projects may have had somewhat different objectives regarding 
desired changes in instructional practice, all sites reported changes in instructional 
practice as a result of the advent of technology at the school. 

• All schools reported the increased use of individual instruction, cooperative learning, and 
student-based projects and less use of direct instruction and lecturing. (1) 

• Over 75% of the respondents to the teacher survey reported positive changes affecting the 
quality and effectiveness of instruction that was attributed to the advent of technology in 
their school. (2) 

Was there increased parent/home involvement? 

All sites reported some form of gain in parent/home involvement. 

• A number of sites reported large increases in the amount of parent- school involvement 
regarding instructional matters. (1) 

• Substantial gains were also reported in parent use of technology at home and in parent 
attendance at school events. (1) 



Was there dissemination of knowledge and/or products? 

All sites engaged in dissemination activities within the project schools, within the 
project districts, to other districts in the state, and in some instances outside the state. 

• The majority of the projects reported that the most dissemination activities to date had 
occurred within the project schools. (1) 

• The projects generally reported between 25 - 75 in-district dissemination activities, 
although one project reported 135 and another reported none. (1) 

• Total visitors to the projects ranged from less than 300 for two of the sites, to between 
500 and 1,000 for three sites, to over 2,500 for the last site. (1) 

• All of the project sites reported that dissemination would be an important focus for the 
fifth year of implementation, although some had a much clearer vision and more solid 
structure in place than others. (5) 

• On the teacher survey for this project, which was disseminated to all teachers at the 
project schools, 45.8% reported that they had developed technology-related applications, 
products or procedures that they had shared with other teachers at their school. In 
addition, 27.3% had shared them with other teachers in the district and 21.3% with 
teachers outside the district. (2) 



Did new business and higher education partnerships result? 

All projects formed relations with business partners, and many also formed 
partnerships with higher education institutions. 

• Project revenues from donations across the six project sites ranged from approximately 
$40,000 to nearly $300,000 during the 1989/90 project year. 

• All of the projects reported increases in the number of business partners gained through 
the projects. (1) 

• Virtually all reported an appropriate fit between business partners and the school agenda. 

( 1 ) 
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• There was mixed reporting about the inclusion of these business partners in active project 
planning and implementation. (1) 



Do teachers and administrators value the programs? 

The vast majority of teachers and administrators responding to the teacher survey and 
whom we interviewed clearly valued this technology program. 

• Of the respondents to the teacher survey who were project participants, 84.9% said they 
were participating by choice. (2) 

• Another 54.5% of the respondents who were not project participants said they would like 
to be. These high levels of interest are despite the fact that the project impinges 
substantially on teacher time. Teachers report spending an average of 18.6 hours per 
week on technology-related activities outside the classroom. Most of this time, 71.5%, is 
not compensated. (2) 

• Regarding future plans for technology use, only 6.7 % of the teacher respondents said 
they do not plan to use or will reduce their technology use. 71.4% responded that they 
intend to expand current use (58.3%) or to start using technology soon (13.1%). (2) 

• Of the respondents to the teacher survey, 75.2% expressed positive support; while 
44.9% indicated significant positive support from the school administration for 
the project. (2). 

Was there equal access for student, teachers and administrators? 

Initial access to project participation was generally open to teachers and 
administrators. However, there are still teachers in project schools who would like to 
participate in the project, but access is now closed to them. While students were 
generally not able to choose participation, per se, the projects were well distributed to 
schools with broad ranges of ethnic minorities and special populations. 

• Across the six projects active teacher and administrator project participation 
averaged over 50%. (1) 

• Of the teachers participating in the project, 84.9% reported participating by 
choice. (2) 

• However, compared to the number of teachers participating in the projects, 
another 30% said they were not in the project, but would like to be. (2) 



8. Recommendations 

What policy guidance can be gained from the MTS Level I experience? 

Based on the data collected, the site visits, and the many interviews conducted in 
conjunction with this project, several changes in state policy regarding MTS Level I- 
type projects are warranted in the opinion of the evaluation team. 

• State goals and objectives should be set in advance of the initiation of the project. These 
could be sufficiently broad to allow for considerable local discretion in implementation 
but should be sufficiently clear to allow common evaluative criteria to be established 
prior to the onset of the project. 

• Prior to entry into this type of a continuing grant program for project implementation, 
there should be a planning phase for each project. 
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• If there is to be a local evaluation component to the project, this should be designed by 
each local project as part of the planning process. It should be designed to best meet local 
needs and to fit into the overall local project design. 

• Recipients for implementation grants would be based on the plans resulting from the 
planning grants. Certain district assurances of support should accompany these plans. 
These should include assurances that the leadership and staff of each proposed project 
school have been actively engaged in the planning process to date, and are committed to 
the implementation of the project as it has been designed for that school. 

• To the greatest extent possible, the projects should be kept on a well-established time 
schedule that clearly allows them to plan in the first year, have the desired equipment on 
hand at the start of the second year so that training can take place, sufficient 
implementation by the third year to ensure that site visitors will have something in place 
to see, and product and service dissemination preparation in the fourth year, with actual 
dissemination of products and services occurring in year five. 

• High-powered Level I MTS type projects make little sense from a state perspective 
without a dissemination emphasis. The state will need to provide more guidance and 
central assistance regarding the shape, form and logistics for these dissemination efforts. 

• Annual central assessment processes should be in place for the duration of the project. 

One key role of this centralized assessment team should be to assess whether each project 
is on course to meet state objectives, to signal any mid- course corrections that may be 
needed, and to allow a check on the continuing allocation of state funds in extreme cases 
of dysfunction. 

To what extent do the MTS projects provide models for the use of technology to support and 
expand the curriculum frameworks ? 

• There is considerable potential in this regard. All of the Level I MTS projects have set 
this as a goal, and have achieved varying levels of success in developing specific 
procedures and curricular materials that are designed to tie technology to the state 
curriculum frameworks. 

What are most effective as demonstration vehicles for conducting research and disseminating 
successful methods? 

• Insufficient research has been conducted in this area. This is clearly the key to judging 
the success of a Level I MTS type project. The state is making a substantial investment 
in a few sites and the return has to be judged in terms of state-wide impact. After touring 
every project school and seeing a number of the products under development, the 
research team is convinced that these two mechanisms alone, i.e., visitations and product 
development will obtain optimal dissemination. 

Should the program be continued, discontinued, or expanded? 

• The program should be continued, but with the modifications described above. 

There should also be three distinct phases for participants: planning, 
implementation, and dissemination grants. Some form of centralized assessment 
should occur annually to ensure accountability. 
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D. Level II Model Technology Schools 

1. Background Information 

Description: The Level II Model Technology. Schools (MTS) program was initiated in April 
1987 when the CDE awarded AB 803 grants to ten projects to combine curriculum improvement 
with the integration of technology within a single curriculum area across several grade levels at a 
school site. Funding for six project sites was continued under AB 1470 to package the programs 
for dissemination throughout the state for other schools to adopt or adapt the models. 

Status: The six projects currently funded are: Project LINKS (elementary English-language 
arts), Project FUTURE (middle school English-language arts), Project HAT (middle school 
history-social science), Project TIME (high school history-social science), Project TOPS 
(elementary science), and Project TASC II (middle school science). A seventh project, focused 
upon mathematics for grades K-6, was just added for dissemination during 1991-92 but is not 
included in this report. 

Data Sources: The key to the major data sources used for the assessment of the software 
development projects is as follows: 1 = Self-Assessment Inventory, 2 = teacher survey, 3 = 
student survey, 4 = information from site visits and telephone interviews, and 5 = observations 
and judgements of CETAP staff. Data from these sources are described in detail in Volume II of 
the CETAP Phase III report. 

2. Planning 

To what extent were there changes in site -based planning for technology 

All of the Level II MTS projects experienced major changes in program planning. 

• All six projects reported significant changes in their planning procedures, and all 
agreed that flexibility was necessary to the success of the projects. (1) 

The CDE requirements to operate the Level II MTS projects over a five-year period 
greatly facilitated inclusion of technology into school-site planning and integration with 
existing site committees. 

• Four of the six schools used committees for initial and on-going project planning 
and coordinated planninjg activities with other committees such as SIP, site 
council, and school curriculum. (1) 

• Two projects without planning committees relied on the judgement of project 
directors with input from other staff members. (1) 

• Five of the projects eventually developed district and/or site level technology use 
plans. The sixth site is in the process of developing a district plan. (1) 



3. Content 

To what extent did the Level II MTS projects support the California curriculum frameworks? 

The planning, development, and implementation of Level II MTS projects established 
models for the infusion of technology into the state curriculum frameworks. 
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• The subject matter emphasis of the Level II MTS projects encouraged systematic 
integration of technology with curriculum as outlined in the state frameworks and 
increased teacher implementation of the curriculum frameworks (2). 

• The emphasis of the Level II MTS projects on using the curriculum frameworks 
as they implement technology use has prompted adopting schools to replicate this 
process. (2) 



What types of technology were typically incorporated in the Level II MTS projects? 

Most projects emphasized computer use and some applications of other technologies 
along with laboratory settings to integrate technology into the curriculum. 

• While computer use received the greatest emphasis among the Level II MTS 
projects all of them integrated a variety of technologies with the curriculum, 
including: computers, ITV laser disks, CD-ROM, VCRs, LCD display panels, 
camcorders, and telecommunications. (1) 

• The six projects use lab settings to a varying degree for the integration of 
technology with the curriculum. Labs are often used in conjunction with 
classroom activities. 

• One Level II MTS project used a lab setting to house technology equipment 
mounted on movable carts for teacher check-out (for classroom use) and to 
conduct staff in-services in use of various hardware and materials. (1). 

• Another project used classroom mini labs of 10-12 machines each. 



To what extent was instruction expanded and/or improved as a result of Level II MTS project 
goals and integration of technology into the curriculum? 

Technology integration greatly expanded teacher instruction, particularly emphasizing 
a shift from teacher-directed instruction to student-centered instruction. 

• All six project directors reported technology expanded instruction far beyond 
what could have been achieved without it. (1) 

• Technology greatly facilitated student-centered instruction, allowing teachers to 
facilitate learning instead of just providing information. (1) 



4. Implementation 

To what extent was staff development supporting Level II MTS program implementation 
provided? 

The projects all provided staff development to match a broad range of staff needs both 
during implementation of the project and on an on-going basis. 

• Intensive staff development was required to implement the projects and was also 
necessary on an on-going basis to help teachers integrate technology with 
classroom teaching. (1) 

To what extent were the major activities implemented as planned? 

In most instances, the implementation of Level II MTS activities extended beyond the 
dates originally planned. In addition, many changes were made to the activities during 
the course of the projects. 
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• Several project directors cited delays in the delivery of equipment and resources 
as a major impediment to achieving project goals that required constant 
modifications to original planning (1). 

• Many directors reported having to rely on advice and direction from sales 
representatives, which often was erroneous or not in the best interest of the 
project, thus causing additional delays in implementation (1). 

• Adjustments in planning to accommodate the use of reduced equipment and/or 
resources caused numerous delays in implementation (1). 

• Keeping up to date with advances in technology was difficult and costly and 
equipment was often delivered late. (1) 

• In many instances, projects had to adjust original plans to accommodate reduced 
equipment/resources as original costs escalated. 



To what extent was project implementation coordinated with other programs! agencies? 

Collaboration with other programs, other agencies, and other Level II MTS projects 
was consistently reported as frequent and beneficial. 

• Assistance with implementation from CTP, CDE, ITV agencies, California 
Literature Project, and other agencies (county offices, etc.) was reported as 
beneficial and helpful. These agencies also were of assistance in dissemination 
activities (1). 

To what extent have the Level II MTS projects conducted evaluation activities? 

Evaluation activities have been implemented on a limited basis. 

• A variety of formative and summative evaluation measures were administered at 
all six projects, but there were not consistent or standardized measures 
administered across sites. (1) 

• The directors stated that more outside support was needed in order to evaluate the 
projects effectively. (1) 

• Workshops and other dissemination activities were evaluated with surveys by 
most of the projects during 1990-91. (1) 

• Teacher/student observations, portfolios (both written and videotaped), and 
analysis of standardized test scores were used with little or no consistency as 
evaluative measures (1) 

• Most projects relied on verbal exchanges among faculty members to provide 
information regarding project success and effectiveness. (1) 



To what extent was there equal access for students, teachers, and administrators? 

While the allocation of CDE funds to all six Level II Model Technology Schools was 
equitable, they were not equal in terms of several other important factors. 

• Curriculum Emphasis'. Through the end of 1990-91, only four major curriculum 
areas were addressed by Level II MTS projects; none of the projects focused upon 
mathematics or foreign languages and the one that addressed visual and 
performing arts was not continued under AB 1470. (1) 

• Grade Level Emphasis'. Most of the Level II MTS projects focused upon the 
elementary and middle grades. Only one project addressed the grade 9-12 
curriculum in history-social science. (1) 
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• Staff Stability : The Level II MTS schools were specifically charged to develop 
the capabilities to provide dissemination services to AB 1470 school-based sites, 
and others, as requested. Four of the Level II MTS schools had the same project 
director since the beginning of funding four years ago. The director of one project 
resigned after three years and has been replaced by co-directors. The director of 
another project died during the fourth year of the project and was replaced with an 
interim director who in turn, was replaced with the new director at the beginning 
of the 1991-92 school year. 

• Staff Responsibility: Three project directors have no classroom responsibilities, 
and their sole assignment is to the project. Three schools required project 
directors to continue classroom assignments either full or part time. This directly 
affected the ability of directors to disseminate project activities (in one instance, 
no dissemination activities were conducted during the last year). 

• "Adoptability” : Three of the Level II MTS projects (two language arts, one 
science) have had a large number of schools indicate they are adopting or 
adapting their projects. The other three have had moderate to few indications of 
adoption/adaptation. 



5. Project Dissemination 

To what extent did the Level II MTS projects assist other schools to implement the use of 
technology in their programs? 

Level II MTS schools were able to assist other schools implement technology into their 
programs on a varied and generally limited basis. 

• All six projects utilized a variety of marketing techniques including: brochures, 
booklets, teleconferences, presentations, workshops, seminars, visitation tours, 
journal articles, etc. (1) 

• All directors indicated that additional assistance was needed from various 
agencies, such as the CDE and the California Technology Project, in order to 
successfully market their projects’ programs and practice to other schools. (1) 

• Some of the projects attempted to establish formal adoption agreements with other 
sites but were unsuccessful due to the lack of time and resources to plan and 
establish the level of involvement needed for formal agreements. (1) 

• Time and funding constraints severely limited adoption follow-up activities. (1) 

• All project directors agreed that conducting visitations was the best means of 
disseminating programs and practices to other schools. Person-to-person 
consultation was also very effective. (1) 

• A total of 172 schools adopted or adapted Level II MTS projects. Eighty percent 
of these were AB 1470 School-Based Grant recipients. In all, 19 percent of the 
School-Based Grant projects adopted or adapted Level II MTS programs and 
practices. (1) 

• The quality and level of adoptions of projects was not possible to determine. 

• Many schools resisted signing formal agreements. 

• Formation of the Academic Model Technology Coalition (AMTEC), a coalition 
of the six Level II MTS projects, facilitated collaborative dissemination of Level 
II MTS projects. 
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6. Project Support Resources 

To what extent did new business and higher education partnerships result? 

A major equipment and software donation was received from IBM. 

• All six Level II MTS projects formed a coalition, AMTEC, to support 
collaboration in seeking additional grants and business partnerships. (1) 

• As a result of the AMTEC coalition, the projects have received approximately 
$500,000 in equipment and software donations from IBM. Other contributions 
are expected in the future. (1) 

To what extent has the IBM equipment donation affected the projects? 

The directors suggest that the donation will be a valuable resource, but some difficulties 

have been encountered in utilizing the equipment. 

• Each site received approximately $80,000 in equipment, software, and training. 

( 1 ) 

• Directors expressed some frustration that promised delivery and training dates 
were not kept and were in fact severely delayed. (1) 

• The project directors have some concerns about the difficulties that will be 
encountered by staff and students in learning to operate the equipment. Some 
reported the time required for training may not be cost-effective. (1) 

To what extent was assistance provided by the CDE, CTP, and other agencies? 

Significant assistance was provided to the MTS Level II projects by the CDE. 

• All six project directors stated that the assistance provided by the CTP, CDE, 
county offices. Subject Matter Professional Development Projects, and ITV 
agencies was beneficial. In all instances, there was a request for continued 
support and assistance from these sources. (1) 

• The CDE provided additional funding to Project FUTURE for the production of 
videotapes. 

• The CDE Office of Educational Technology staff worked closely with Level II 
MTS Directors to assist in planning and packaging programs for state-wide 
dissemination. 

To what extent was there increased parent! home involvement? 

Parents helped with the implementation of one project. 

• At Project TOPS, parents provided assistance with various school-wide science 
enrichment activities and events (i.e. science fair, assemblies, lab). 



7. Funding Support 

To what extent was additional funding leveraged by the projects? 

Extensive funding was received from outside (non-AB 1470) sources. 

• Projects were able to double the funding provided by the state by seeking the in- 
kind support and funding from their district office and business and industry. (1) 
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• The largest funding source, other than AB 1470 money provided by the state, was 
matching funds from the district. (1) 

• Other additional funding came from workshop fees, sales of publications, SIP, 
CTP, ITV agencies, technology vendors, and consultation fees. (1) 



8. Cost-Benefits 

To what extent can the programs show cost-benefits? 

The Level II MTS projects provide a cost-effective resource for state- wide integration of 
technology into curriculum. 

• Dissemination of Level II MTS projects reduces the need for other schools to 
invest in costly trial and error efforts to integrate technology. Pre-packaged 
curriculum integration solutions can be provided. (1) 

• Teacher involvement in adapting Level II MTS projects may serve as an efficient 
method to encourage framework implementation. (1) 

• School-based grant applicants who adopt Level II MTS projects can receive staff 
development and other services which have been proven to be effective, ensuring 
that funding will not be wasted on ideas that don’t work. (1) 



9. Supporting and Impeding Factors 

To what extent was implementation supported? 

The projects were greatly supported by funding from the CDE and assistance from 
other agencies. 

• CDE funding greatly facilitated the integration of technology into the curriculum. 
Without this funding, little if anything would have been undertaken at the project 
sites. (1) 

• District office support was essential to Level II MTS project success. (1) 

• Support from agencies such as the CTP, county offices of education and CDE 
were consistently cited by the project directors as having greatly facilitated 
implementation. (1) 

To what extent was implementation impeded and why? 

Lack of time and funding and consistent guidelines for dissemination efforts was a 
major impediment to project implementation. 

• Project directors often had teaching responsibilities which interfered with 
dissemination and other project activities. (1) 

• The loss of Supplemental Grant funds by districts receiving AB 1470 Level II 
MTS grants greatly impeded both project effectiveness and dissemination efforts. 

( 1 ) 

• Travel expenses impeded dissemination efforts, making it difficult for projects to 
cover the entire state. (1) 

• Finding appropriate training resources was always a problem. (1). 

• Insufficient clerical help created many “paper burdens” for directors. (1) 
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• Expectations provided by the CDE about dissemination and criteria for acceptable 
“adoptions” were unclear and inconsistent. 



10. Project Outcomes 

To what extent did student performance (academic and behavioral) improve? 

Major gains in student attitude, interest, academic performance, and technology use 
were observed and reported. 

• All directors and most project teachers indicated student interest and motivation 
were greatly improved by the addition of technology to the curriculum. (1) 

• Students became much more more engaged as evidenced by increased 
participation in activities and problem solving activities. (1,2) 

• Students were better able to work in cooperative learning groups. (1,2) 

• Though the major emphasis was on using computers for writing, students were 
able to use a variety of technologies on a frequent basis. (1,2) 

To what extent were there desired changes in instructional practices ( teacher performance)? 

A variety of changes in teaching style at Level II Project sites were reported to be 
favorably influenced. 

• Teachers became comfortable with using technology in their classes. Before the 
Level II projects, many teachers were intimidated by technology. (3) 

• Teachers indicated they learned about technology from their students. (3) 

• Many teachers reported their presentation skills increased as a result of the 
project. (3) 

• Communication and collaboration among teachers was increased. (3) 

• Teacher familiarity with the curriculum frameworks and willingness and ability to 
align instruction with them was increased. (3) 

• The use of alternative assessment techniques was increased as student 
performance was easier to track on the computer. (3) 

To what extent were unanticipated outcomes assessed? 

A number of unanticipated outcomes were reported by the project directors. 

• At three of the project sites, technology use increased in other subject areas in 
addition to the one targeted. (1) 

• The level of interest in adopting practices and products was under anticipated for 
some projects. (1) 

• Improvements in student motivation were beyond expectation. (1) 

To what extent do teachers and administrators value the projects? 

The participating project directors and most of the teachers expressed a great deal of 
enthusiasm towards the projects. 

• Project directors report that project activities produced significant benefits and 
were definitely worth the effort put into them. 
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• Project teachers report technology significantly improved student interest and 
motivation and allows them to become pro-active learners. 



11. Recommendations 

To what extent should the projects be continued, discontinued, modified or expanded? 

The Level II MTS projects should be continued with refinements in the products, 
practices and in the dissemination procedures. 

• The projects should continue to be funded after 1991-92 with the condition that 
dissemination plans be developed and approved. 

• To support state-wide dissemination, the funding level should be increased. 
Additional funding could also be used to develop and disseminate integration 
strategies for new and emerging technologies and to hire full-time project 
directors. 

• Additional exemplary projects should be identified and funded for dissemination 
in other subject areas. 

• An expanded Level II MTS program could include schools in more diversely 
representative geographic areas. The current schools are all located in suburban 
settings. 

• The emphasis on planning should be continued in the future for Level II MTS 
projects. 

• Continue to require Level II MTS emphasis on integration of technology into the 
curriculum according to the state curriculum frameworks as a successful means of 
assisting teachers to better understand, apply, implement and use them fully. 

• Maintain staff development as a required and fundamental component of project 
implementation requirements. 

• Projects should define and disseminate successful procedures for planning and 
infusing technology into instructional programs. 



What improvements should be made? 

All aspects of the dissemination/adoption process need to be reviewed and revised to 
increase the desired impact of the Level II MTS Projects. 

• Evaluation requirements should be increased and standardized across all six 
projects. In addition, project adopters should be evaluated for levels of adoption 
and impact on students and staff. 

• Standardized, field tested evaluation forms should be provided, along with expert 
outside assistance to facilitate evaluation. 

• Collaboration with the CTP and other agencies should be planned and increased. 

• Staff development should be provided for district administrators to improve their 
awareness and support of the projects. 

• Formal planning committees should be required for Level II MTS funding, along 
with district office planning involvement for maximum project effectiveness and 
for integration of technology into on-going district/site goals. 

• Several exemplary School-Based Grant adoption projects should be selected and 
funded to serve as remote demonstration sites to assist dissemination. 
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• Level II MTS projects should develop and disseminate sample Technology Use 
Plans and “exemplars” for infusing technology into the curriculum. 

• Identify and establish procedures for dissemination of project practices and 
products. 

• Establish minimum criteria for schools to meet when adapting or adopting a Level 
II MTS project. 



E. Instructional Television Agencies 

1. Background Information 

Description: Instructional video and support services are provided for schools in California by 
regional ITV agencies. The seven agencies work in conjunction with the CDE’s Office of 
Educational Technology and the media and technology staff in the counties and districts served. 
Financial support for ITV agencies is provided through an AB 1470 grant, host county offices of 
education and local districts, public television stations and in some cases, membership fees. 
Instructional video programming is selected by curriculum committees of teachers and 
administrators representing schools and local educational agencies. The ITV agencies also play a 
major role in the evaluation and distribution of ITV programming and in staff development 
programs on video resources throughout the state. 

Status: The seven ITV agencies currently funded by AB 1470 are: the Northern Instructional 
Television Advisory Council (NITAC), KQED-ITV, Santa Clara County EMC-ITV, the 
Regional Educational Television Advisory Council (RETAC), Los Angeles Unified School 
District (KLCS), Telecommunication of Orange County (TOC-KOCE), and the San Diego 
County Office of Education. These agencies all receive AB 1470 grants of $.40 per student 
served. 

Data Sources : The key to the major data sources used for the assessment of the Instructional 
Television Agencies is as follows: 1 = Self-Assessment Inventories completed by agency 
directors in 1990-91, 2 = Self-Assessment Inventories for 1989-90, 3 = analysis of the 
Corporation for Public Broadcasting’s (CPB) Study of the School Uses of Television and Video, 

4 = survey of media directors, and 5 = CDE’s 1989 ITV Study. Data from these sources are 
described in detail in Volume V of the CETAP Phase II report. 

2. Planning 

To what extent are the needs of schools considered in ITV agency planning? 

There is considerable variation among the ITV agencies in capability to assess the needs 
of schools for instructional television programming and support services. 

• Some agencies do not utilize systematic means to assess the needs of schools for 
ITV programming and support services. Most agencies that do conduct formal 
teacher surveys of ITV utilization get limited returns. (1) 
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3. Program Content and Implementation 

To what extent do teachers prefer to use off-air broadcasts of ITV series in comparison with 
prerecorded programming on videotape? 

There is a discrepancy between the preferences of teachers for using regular daytime 
broadcasts of ITV programs and the reports of instructional media specialists about the 
emerging dominance of videotaped programming. 

• There are no reliable state-wide data on the proportions of teachers who use off- 
air broadcasts of ITV programs in comparison with prerecorded series, but both 
EMC-ITV and KLCS report survey results that demonstrate a clear preference 
among elementary school teachers (up to 75%) for using regular daily broadcasts 
of ITV programs over videotaped versions. 

• Nine out of ten LEA media directors reported that the distribution of ITV 
programming on videotapes was both highly utilized and considered very 
valuable by the schools in their service areas. Over a third suggested (in 
responses to an open-ended question about how to make ITV service delivery 
more effective) that local tape distribution was more efficient than broadcast. (4) 

To what extent did the ITV agencies support the state curriculum frameworks? 

Larger ITV agencies are able to allocate resources to develop school resource materials 
on using instructional video to align video programming with the state curriculum 
frameworks. 

• KQED-ITV prepared and distributed ITV Curriculum Correlations for five of the 
major curriculum areas: history-social science, English-language arts, 
mathematics, science and the visual and performing arts. (2) 

• RETAC distributes (with state augmentation funding) Reading, Literature and 
Television: A Powerful Partnership , a publication detailing ways to use 
instructional video to implement the English-Language Arts Framework for 
California Public Schools, that was produced cooperatively by the California 
Literature Project, California Instructional Video Consortium (CIVC), and the 
CDE’s Office of Educational Technology. (2) 



4. Funding Resources and Constraints 

To what extent do state funds enable the ITV agencies to attain their program outcomes? 

There is considerable variation among the regional ITV agencies in the proportions of 
their operating budgets provided by the state ITV grants, membership fees, and local 
agencies. 

• While CDE grants to the ITV regional agencies average about one-third of their 
annual budgets (37.4%), the range is from over half (KQED-ITV, 51.3%) to less 
than one-fifth (TOC/KOCE, 18.3%). (1) 

• The ITV agencies vary significantly in terms of non-state sources of revenue. 

RETAC, for example, receives 47% of its revenue from membership fees and 
nothing from its host agency while the Santa Clara EMC-ITV receives 24% from 
membership fees and 53% from host agency support. (1) 
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5. Outcomes 

To what extent did the ITV agencies contribute to: desired changes in district and site-based 
planning to use ITV; and in actual teacher use of television as an instructional resource? 

While instructional video is generally included in district and school-site planning for 
instructional improvement, it is difficult to determine the specific contributions of ITV 
agencies versus other resources at the school level. 

• Nearly three-quarters of all district superintendents and school principals report 
that ITV was included as a part of instructional planning. One district and one 
school reported that ITV was included “extensively” in planning for curriculum 
improvement. (3) 

• More than half of the teachers responding to the CPB Study of the School Uses of 
Television and Video (56.9) were not sure whether their schools received ITV 
support services from one of the regional agencies. (3) 

• More than a third (37.3%) of the teachers reported they had not received any 
information about ITV during the school year, and over a quarter (27.7%) were 
not sure if they had received any information or not. (3) 

• The ITV agencies estimate that a majority of California teachers make at least 
some use of instructional video in their teaching. This is confirmed by 57% of the 
teachers in the CPB study. (1) 

• 43% reported they did not use ITV services (3) 

To what extent is there agreement among educational technology specialists in the state about 
the need to support the system of state-supported regional ITV agencies? 

Though educational media center directors hold widely divergent views on the levels of 
state support needed for regional ITV agencies to provide instructional video support 
services, high overall value was placed on the agencies. 

• Nearly two-thirds of the county office and district media department directors 
(64.9%) responding to a CETAP survey reported that they would like to see the 
present system of state support for ITV agency services changed and over half of 
them (54. 1%) suggested state support for the county offices to provide video 
services. (4) 

• Most of the county office and district media department directors surveyed gave 
generally high ratings to the services provided by the ITV regional agencies. (3) 

• Approximately a third of the media directors (38%) reported that local distribution 
of videotapes was more efficient than broadcasting programs, while 62% 
supported the use of broadcast. 

• Only 24% of the media directors indicated a preference for reallocating the state 
ITV grant funds. (3) 

• There was no consensus among the media directors about options for broadening 
the scope of ITV agency services to include other, emerging educational 
technologies, such as multimedia. (3) 



To what extent did the ITV agencies implement the CDE goals and initiatives? 

The seven ITV proposals vary widely in the quality and quantity of attention given to 
the program goals and priorities specified in the CDE funding application guidelines. 
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• Except for NITAC, which relies on media center staff in each participating 
county, all of the ITV agencies provide at least some staff development activities 
to local education agencies in the effective use of instructional video. (1) 

• During 1989-90, all of the ITV agencies provided support to AB 1470 School- 
Based Educational Technology projects in planning and implementing activities 
involving instructional video. (2) 

• Eleven percent of the School-Based Grant projects reported that ITV was a major 
emphasis in their project objectives, 24% noted secondary emphasis, and 64% 
indicated no emphasis. (1) 

• Of the seven recommendations provided in the final report of th t California 
Education Summit: Meeting the Challenge, the Schools Respond, the ITV 
agencies indicated that they were all adapting the general idea of “enhancement of 
the curriculum” in agency program planning. Three agencies reported some effort 
to support all of the Summit ideas but the others considered one or more of the six 
recommendations to be beyond the scope of ITV agency capabilities. (1) 

• The ITV agencies reported generally high levels of collaboration with various other state- 

supported educational reform and curriculum improvement programs except for the 

regional consortia of the SB 1882 Staff Development Program. (1) 

To what extent was there equal access to ITV for students, teachers, and administrators? 

State ITV grants help compensate for the unequal access to ITV support services of 
schools in different parts of the state but do not overcome all of the inequities. 

• The disparities in local resources to support basic ITV services and remain current 
with emerging technologies are compensated for to a limited extent by the state 
grants to the seven agencies. (1) 

• Schools which do not subscribe to instructional video services in the three ITV 
regions charging membership fees do not receive the same level of support 
services as do the schools that can afford to pay regional agency membership fees. 

• The schools in the ITV regions with agencies that do not have staff personnel to 
provide ITV utilization workshops as part of staff development services do not 
receive the same level of support services as the schools in the other regions. (1) 



6. Recommendations 



To what extent should the program be continued, discontinued, expanded or modified? 

ITV services should be continued but within a structure that increases equal access and 
level of use of ITV in conjunction with other technologies. 

To reduce the duplication of effort among the existing ITV agencies: 

• Identify the common and unique services of each agency. Then, develop a 
structure whereby agencies could significantly increase sharing of resources 
across agencies to avoid duplication. 

• Shift ITV agency funding now devoted to the publication of regional ITV 
schedules, catalogs, curriculum alignment guides, and other print resources to a 
single agency to produce a state-wide instructional video resource guide that 
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would be maintained on-line in the TRIE database and distributed to schools each 
year in a print version. 



To meet the changing needs of schools and teachers: 

• Periodically conduct sample surveys of California teachers ( not districts or media 
directors) about technology use to assess school needs for ITV programming and 
support services. 

• Study the final results of the CPB’s Study of the School Uses of Television and 
Video when they become available to determine state-wide patterns of ITV use by 
teachers at different grade levels to determine if the survey results in San Jose and 
Los Angeles are representative of the situation throughout the state. 

• ITV agencies should broaden their scopes of service to include other technologies 
such as distance learning and the use of video teleconferences to increase access 
to staff development. 

To better integrate ITV into the curriculum: 

• Support the production and distribution of additional publications similar to 
Reading, Literature and Television: A Powerful Partnership to support the 
implementation of the other subject area frameworks. 

• Increase the dissemination of information about the availability of ITV agency 
services, state-licensed ITV series, and of the rights of schools to tape 
programming and establish school-site video libraries to support instruction. 

• Integrate exemplary instructional video use into other educational initiatives: each 
of the major curriculum areas, compensatory education programs, and various 
school improvement and restructuring programs. 

To insure coordination of state efforts: 

• Revise and update the CDE Application Guidelines for the ITV regional agencies 
to ensure (1) that the annual operating plans of the ITV regional agencies propose 
activities that are consistent with current CDE initiatives in curriculum, learning 
resource management, staff development, and other AB 1470 programs, and (2) 
that the agencies will employ uniform techniques to evaluate the effects of their 
programs. 

• Establish formal procedures to increase collaboration between the ITV agencies 
and the Curriculum Framework and Textbook Development Unit, the Subject 
Matter Professional Development Projects, the SB 1882 Staff Development 
regional consortia, and the CTP. 

• Evaluate current CDE policies for funding the regional ITV agencies in the 
context of equal access to instructional video programming and support services 
and make adjustments as necessary to ensure equity for all students. In particular, 
the evaluation process should address the needs of schools in remote areas of the 
state and those experiencing rapid population growth. 

• Develop cross-agency procedures to coordinate production by the ITV agencies 
and/or independent television producers of video programming that supports the 
California curriculum frameworks. 
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F. California Technology Project 

1 . Background information 

Description: The California Technology Project (CTP) consists of a Central Office and 14 
regional consortia. The CTP was originally initiated in 1988 with AB 803 funding and continues 
to be funded under the provisions of AB 1470. The Central CTP coordinates all aspects of the 
CTP, and major state-wide activities can be organized into four categories: state-wide leadership 
and support for the regional consortia, information collection, information dissemination, and 
staff development. The Central CTP supervises the regional consortia, overseeing the 
development of their by-laws, budgets, governance, and programs. The major functions of the 
14 Regional Consortia are to: provide staff development in technology integration; assist in AB 
1470 project development, implementation and evaluation; coordinate resources for technology 
use; facilitate the use of TRIE; and provide individualized technical assistance to schools served 
by the consortium. 

Status: Currently there are fourteen regional consortia which, together, serve much of the state. 
Generally, their regional lines roughly follow along county lines, but in some cases, there is 
overlap. Most often, they serve multiple counties. The areas served by the CTPs are member 
driven and based on local needs. 

Data Sources: The key to the major data sources used for the assessment of the California 
Technology Project is as follows: 1 = Self-Assessment Inventories completed by the consortia 
coordinators and the Central Office, 2 = user survey, 3 = phone interview and 4 = comments and 
surveys from other projects. Data from these sources is described in detail in Volume IV of the 
CETAP Phase II report. 



2. Planning 

To what extent were consortia members involved in planning? 

All consortia engage in planning processes that are based on client needs as ascertained 
through formal needs assessments, informal contact and the Consortia Council and/or 
planning committee. 

• The regional consortia developed annual plans which served to guide their day-to- 
day operations. (1) 

• Ten of the consortia have a formal planning group that is separate from the 
regional council. (1) 

• All of the CTP consortia conducted needs assessment surveys of members, and 
eight used additional input from staff judgement, the council, and other informal 
sources. (1) 

• There is a great deal of coordination with the host agencies in both planning and 
delivery of services. (1) 

To what extent were CDE initiatives addressed in regional plans? 

All of the regional consortia addressed all five CDE initiatives relative to local needs. 

• The CDE initiative given the highest priority by the consortia was staff 
development. Second was school-based technology use planning; third was 
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curriculum alignment; fourth was evaluation and accountability; and fifth was 
learning resources management. (1) 

• In the first year of operation (1989-90), the needs most often identified were: 
technology in the curriculum, administrator training, teacher training, proposal 
writing assistance, and technology use planning. (1) 

• Similar needs were identified for the second year, with the addition of AB 1470 
School-Based Grant implementation and multimedia usage. (1) 

Were there changes in site-based planning for technology? 

The CTP was the prime delivery system for information about the School-Based Grants 
which catalyzed focused planning at school sites throughout the state. 

• The consortia generally followed a pattern of designing their staff development 
around supporting the planning and proposal writing for AB 1470 School-Based 
Grants. (1) 

• The CTP publicized the grants and then provided training in developing and 
preparing proposals in response to the RFP. Almost every regional consortium 
reported that site-based technology use planning was a moderately high to high 
service priority. ( 1 ) 

• Over half of the responding CTP consortia clients indicated they applied for the 
school-based grants and attended AB 1470 grant writing workshops and had used 
CTP services while completing their proposals. (2) 

• During the telephone interviews every School-Based Grant recipient reported 
significant levels of technology use planning and noted a high degree of 
technology integration into the curriculum. The application process, combined 
with the support received from the CTP, was credited with this improvement. (3) 



3. Program Content and Implementation 

Has the CTP accomplished its major objectives? 

The CTP successfully met its objectives by the second year of operation. 

• 1989-90 was the start-up year, and some of the consortia were not able to 
implement or emphasize some of the activities and objectives. (1) 

• During the second year, most of the consortia were able to fully implement most 
activities, and many exceeded their expectations. (1) 

• Among the staff activities receiving the greatest emphasis were: general 
administration, facilitating the exchange of information, supporting integration of 
technology with the frameworks, soliciting input from teachers, attending 
meetings called by the Central CTP, providing coordination between other 
regional service agencies, assisting the schools reviewing School-Based Grants, 
conducting annual Technology Leadership Academies, and hosting downlink sites 
for annual CTP satellite broadcasts. (1) 

• Fourteen regional consortia are now distributed across the state. Each regional 
consortium provides essential support to schools and districts as they integrate 
technology into their programs. (1) 

• The central CTP established TRIE, initiated partnerships with Subject Matter 
Professional Development Projects, set up electronic conferencing for teachers, 
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produced reports in technology surveys, organized two trained AB 1470 grant 
reviewers, published a quarterly journal, and produced resource materials to assist 
technology use planning and evaluation. 

Did the project support the state curriculum frameworks? 

Both the Central CTP and the regional consortia conduct programs designed to 
integrate technology into the curriculum. 

• The CTP Central office reported cooperative activities with the Subject Matter 
Professional Development Projects including the California Mathematics Project, 
the California Literature Project, and the California Writing Project. (1) 

• The regional consortia offered extensive training opportunities (Teacher 
Leadership Academies) in incorporating technology into the curriculum areas as 
well as into the individual school plans. (1) 

Was staff development that supported implementation provided? 

All of the regional consortia provided client-based staff development which was 
generally well received. 

• During 1989-90, approximately 200 technology-specific workshops were offered 
state-wide, attended by over 7,000 educators. (1) 

• Most of the workshops were 1/2 day or less in length, and many were co- 
sponsored by the host agencies which allowed participants to easily earn in- 
service credit. (1) 

• University partners played an important role in the longer in-service trainings 
which were of appropriate length for university credit. (1) 

• Supporting technology user groups was an important area of emphasis. (1) 

• Educators placed the highest value on the TLA’s and the AB 1470 proposal 
writing workshops. (2) 



Were the major activities implemented as planned? 

Most planned CTP consortia activities were implemented. 

• Impediments during the first year tended to be related to start-up delays and the 
development of internal procedures. (1) 

• By the second year, the regional consortia were able to implement most of their 
planned activities. (1) 

Has the CTP assisted in the dissemination of MTS Level I and II projects? 

The regional consortia provided assistance and disseminated products and practices of 
the Level II MTS projects and less consistent collaboration with the Level I MTS 
projects. There was regional variation relative to the location of the MTS sites. 

• There was collaboration in presenting the summer TLAs and workshops. (1) 

• MTS directors served on CTP planning committees. (1) 

• MTS directors were co-presenters at grant-writing workshops. (1) 

• MTS projects were published in the CTP Quarterly and used TRIE for 
dissemination. (1) 
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4. Marketing 

To what extent was there dissemination of information and/or products? 

The CTP employed a variety of marketing approaches to advertise its services and 
disseminate information. The survey indicated a high level of awareness of specific 
services available by educators who know about the CTP consortia of the CTP. 

• Information dissemination was one of the four main functions of the Central CTP. 
Dissemination was accomplished through print materials, telecommunications 
and other electronic means, as well as staff presentations at conferences 
throughout the state and beyond. 

• Overall, those that used the regional consortia learned of its existence early on - 
about 85% were aware of the consortia within the first 18 months. (2) 

• Over two-thirds of CTP users surveyed reported they were aware of the full-range 
of services the CTP could provide. (2) 

• Of the recipients of the School-Based Grants, 87% were aware of CTP services 
and 68% had used regional services in the preparation of their proposal. (2) 

• The consortia disseminated over 1000 sets of guidelines for AB 1470 School- 
Based Grants. (1) 

• The regional coordinators rated information dissemination as a high priority. (1) 

• The consortia provided assistance and information to schools and districts on 
hardware and software/video. 

• The CTP Quarterly is a source of information on funding, practices, and other 
resources. 



What was the impact of marketing dissemination strategies? 

The regional consortia used both written communication in the form of brochures, 
newsletters and catalogs as well as personal contact. Personal contact was reported as 
most effective by both clients and coordinators. 

• The coordinators reported a high level of direct contact (letters, phone calls, 
personal contact) with clients to inform them about CTP services. This marketing 
method was seen by both clients and coordinators as having the greatest impact on 
awareness. (1) 

• Supporting technology user groups generated client interest in the CTP. (1) 

• The CTP publications were reported as having the lowest impact on informing 
clients of CTP services. (1,2) 



5. Outcomes 

Did CTP activities enhance evaluation activities at the school level? 

The CTP made a contribution to the increase in evaluation activity and sense of 
accountability. Follow-through was seldom possible. 

• The AB 1470 training provided by both the Central CTP and the regional 
consortia included evaluation strategies. 

• Each CTP conducted evaluation workshops attended by some recipients of AB 
1470 School-Based Grants. 
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• The Evaluation Handbook , produced by the CTP, was used to support the training 
and then used extensively by the recipients of the grants. 

Did the program stimulate increased school planning and resource management? 

Clients relied heavily on the CTP for assistance with planning for technology 
integration as evidenced by the high level of participation in the AB 1470 workshops. 

• Educators placed the highest value on the TLA’s and the AB 1470 proposal 
writing workshops. 

( 2 ) 

• The AB 1470 Proposal Writing Workshops had the highest attendance, followed 
by the TLAs, and Technology. Use Planning. (2) 

• Some CTP regions provided special workshops on how to develop a Technology 
Use Plan. (3) 

Was there internal evaluation of CTP activities? 

Internal evaluation among the regional consortia of their activities was inconsistent. 

• Formal evaluations of workshops and TLAs were conducted with the results used 
to guide planning. (1) 

• About half of the consortia evaluated their AB 1470 assistance efforts. (1) 

• Technical assistance and marketing efforts were not evaluated. (1) 



Which services were requested? 

Teachers and school administrators who were aware of the CTP relied heavily on the 
CTP consortia for information and assistance with AB 1470 School-Based Grants and 
for technology use training. 

• Information about the Grant Application Guidelines was requested most often. (2) 

• The CTP services requested often were: AB 1470 project development 
assistance, software/video information/questions, and technical assistance with 
hardware. (2) 

• Teleconferences and ITV staff development were seldom requested. ITV support 
is available from the ITV agencies and/or county office media centers, and 
teleconferences require teachers to be released from classroom duties, making 
teleconferences inaccessible for many teachers. (2) 

• The highest need for increased services (according to both coordinators and users) 
was placed on the expansion of the TLAs and evaluation training for school-based 
grants. (1,2) 

Did the CTP assist schools in the implementation of technology? 

CTP resources greatly facilitated the inclusion of technology into local instructional 
planning and school site plans. 

• Statewide there were: over 1000 requests for assistance with software and video 
selection and technical assistance with hardware; over 600 requests for district 
level technology use planning; and over 700 requests for site level project 
development assistance. (1) 
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Has the CTP assisted in School-Based Educational Technology Grant development and 

implementation? 

The CTP was a major source of assistance to schools developing School-Based 
Educational Technology projects. 

• Of the CTP users surveyed, 58 percent stated their schools had received AB 1470 
grants, and 86 percent of these had been assisted by the CTP. Overall, 81 percent 
of the School-Based Grant project directors had attended a CTP workshop. (2,3) 

• The consortia generally followed a pattern of designing their staff development 
around supporting the planning and proposal writing for AB 1470 School-Based 
Grants. (1) 

• It should be noted that 1989-90 was the application year for the AB 1470 School- 
based Grants Program which accounts for the higher level of requests for grant 
writing assistance. In 1990-91 the requests focused on assistance with 
implementation. In 1989-90 the highest priority was for support of AB 1470 
grant projects. 

• In 1989-90, the major support activity for school-based grants was proposal 
development workshops; in 1990-91 it was technology use workshops and 
evaluation training. (1) 

• Some consortia established telephone “helplines” to assist schools with AB 1470 
proposal writing. (1) 

Was there equal access for teachers and administrators? 

The Central CTP combined with the regional consortia provided services equitably in 
most areas of the state. 

• Most of the CTP offerings were applicable to all clients, but administrator 
specific and teacher specific staff development was offered. (1) 

• Some areas of the state did not have convenient access to CTP services and some 
districts did not belong to a CTP. (1) 

Do teachers and administrators value the program? 

Both teachers and administrators who use the CTP resources and services placed a high 
value on such resources and services. 

• Over 70% of CTP clients wrote in favorable comments on the open-ended section 
of the users survey. (2) 

• CTP clients gave high ratings to the services they had received from the regional 
consortia. (2) 

Has TRIE been used effectively for dissemination of information? 

TRIE has not achieved its potential for disseminating information. 

• The Technology Resources in Education (TRIE) service was provided through 
CSUNet to facilitate dissemination of information on technology resources and 
communication between schools and the consortia. (1) 

• The TRIE service was used by about one-third of the CTP users and only 13 
percent of the School-Based Grant projects. Many stated it was too difficult to 
use. Another factor is the overall low level of use of telecommunications by the 
CTP users surveyed. (2) 
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• Other factors inhibiting the use of TRIE include the lack of modems and very 
limited number of phone lines in the schools. The lack of an 800 toll-free number 
also was an impeding factor for potential users. (2) 

• TRIE received a low level of emphasis during the implementation of consortia 
plans. (2 ) 

• TRIE use was primarily for electronic mail rather than for help in identifying 

technology resources. (2) ' 



6. Resources 

Are CSU and county office of education resources leveraged? 

The CTP consortia received state grants as seed money; all relied heavily on their host 
agency and donations for additional program support. 

• The CTP consortia rely heavily on volunteers and donations. (1) 

• Statewide, it is estimated that in excess of $300,000 of in-kind support was 
generated annually for the regional consortia. (1) 

• Examples of in-kind support included: office space, donated staff time, training 
facilities, use of equipment, college course credit, co-sponsors of workshops, and 
a variety of other items. (1) 

( 

Was project implementation coordinated with other programs? 

There was a great deal of coordination with the host agencies in both planning and 
delivery of services. 

• Coordination generally focused on staff development and technology resources. 

• With regional variation, there was an overall moderately high level of 
collaboration with the ITV agencies. (1) 

• CUE was the most frequently mentioned professional organization collaborating 

with the CTP’ s. (1) 6 

• The level of collaboration with the SB 1882 Regional Staff Development 
Consortia, Educational Telecommunications Network (ETN) was uneven across 
the state. (1) 

• Clients believe that more marketing and visibility are needed, that there is a need 
for better coordination among service providers, and that more funding should go 
into the CTP-efforts. (1) 

Did the project improve the management of learning resources? 

The Central CTP contributed to the management of learning resources through its 
information collection and dissemination efforts. 

• The California Instructional Software and Video Clearinghouses, which provide 
information on K-12 instructional software and CD-ROM programs, help 
California educators select the most appropriate materials to support their plans 
and programs. 

• TRIE furnished educators with evaluative information on model programs and 
practices and effective software and hardware. 
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• The State -of -the -State Technology Survey provided information describing the 
quantity, kinds, locations and uses of technology in a sample of schools in 
California. 

• The Local Assistance Survey provided limited information related to the 
management of learning resources that was specific to the schools that received 
AB 803 Cycle IVB grants in 1987-88. 

Did new business and higher education partnerships result? 

Business and industry collaborators were particularly helpful with donations of 
services, facilities, equipment, and trainers. 

• The consortia coordinators reported an overall moderate level of collaboration 
with business and industry which was perceived as of great assistance to the CTP 
effort. (1) 

To what extent was assistance provided by the CDE? 

The CDE Office of Educational Technology provided assistance to the CTPs to the 
extent possible given budget constraints. 

• The CDE designed the forms, guidelines, and training materials and assisted the 
CTP consortia to provide the Technology Use Planning and Grant Development 
Workshops. 

• The Central CTP office and the CDE collaborated in the delivery of many 
services. (1) 

• Coordinators reported that assistance was usually available by phone from the 
CDE Office of Educational Technology. (1) 

• The CDE provided coordination between the CTPs, MTS projects, ITV agencies, 
and business partnerships. (1) 



7. Allocation of Resources 

How were staff resources allocated? 

Most staff time was allocated for planning, managing, and establishing regional staff 
development activities. 

• The greatest priority for coordinators’ time went to overall planning and 
management. (1) 

• In 1989-90, staff development and workshops took from 7% to 28% of 
coordinators’ time and AB 1470 project support took 17% to 25% of their time. 

( 1 ) 

• 1990-91 shows a slight increase for staff development and workshops which 
required from 12% to 35% of coordinators’ time while AB 1470 project support 
remained at 17% to 25%. (1) 

• In 1989-90, the major support activity for School-Based Grants was proposal 
development workshops; in 1990-91 it was technology use workshops and 
evaluation training. (1) 
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8. Consortium Expenses 



What was the distribution of consortia funding? 

The limited existing data suggest that staffing accounted for most of the expense. 

• Overall, staffing and consultation support were the biggest expenditures for the 
CTP consortia. (1) 

• Only minimal amounts were spent on hardware, almost none on facilities or other 
overhead costs. (1) 



9. Cost Benefits 

Has the CTP been a cost-effective program? 

Both CTP regional coordinators and clients reported the program was cost-effective 
because of the leveraged local support. 

• Almost two-thirds of the CTP’s clients report it is a cost-effective system of 
delivering services. (1) 

• Only 6% of the clients surveyed did not report the program as being cost- 
effective. (1) 

• Given the objectives and expectations of the CTP Regional Consortia, the 
coordinators stated the level of funding provided was insufficient. The funding 
leveraged the acquisition of other funding sources and volunteer support. (1) 

• The largest percentage of staff time allocated from a single resource (37% in 
1989-90; 46% in 1990-91) was provided on a volunteer basis rather than from AB 
1470 funds or host agency funds. (1) 

To what extent does the program show cost benefits? 

The coordinators and clients were in agreement that the CTP is a cost-effective delivery 
system. 

• An analysis of the surveys of CTP clients showed that of those who received AB 
1470 School-Based Grants, 81% had received assistance from the CTP. 

Similarly, the survey of School-Based Grants recipients showed that 86% had 
received assistance from the CTP. (2) 

• The state funding serves to attract in-kind support and donations with the state 
grant, leveraging over $300,000 in support annually. (1,2) 



10. Factors Facilitating or Impeding Sen/ice Delivery 

What were the facilitating factors in the implementation of CTP objectives? 

The major facilitating factor was the support provided by the host agencies and the 
Central CTP. 

• Supporting factors included relationships with the county offices of education, 
universities, and technology vendors. These supporters provided staff 
development, many in-kind services, technical expertise, and support in planning 
and management of the programs. (1) 
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• The Central CTP was a significant supporting factor to the regional consortia. 

The coordinators reported receiving prompt assistance when it was requested. 

Were there factors which impeded the implementation of CTP objectives? 

The major impediments to CTP implementation were size of regions, travel distance, 
and a shortage of funding. 

• The geographic size of the regions was generally an impediment. Travel time was 
a problem in both urban regions (due to traffic) and rural regions (due to distance 
and terrain). (1) 

• Recruiting members (a fiscal necessity) took time away from the delivery of 
services. (1) 

• Mountainous areas have difficulty with telecommunications as well as with travel. 

(1) 



11. Recommendations 

To what extent should the program be continued, expanded or modified? 

The CTP should be continued with several modifications. 

• Establish a CTP Policy Board with representation from CTP users, potential 
regional coordinating agencies, a CTP coordinator, the CDE, the CSU and Central 
CTP that would ensure that all CTP activities are coordinated and address needs 
of all stakeholders. 

• Provide additional assistance and training to CTP coordinators on marketing, 
dissemination, and program evaluation. 

• Expand current efforts to coordinate services with other regional agencies such as 
SB 1882 Consortia, the Subject Matter Professional Development Projects, county 
offices of education, professional organizations, and others, as appropriate. 

• Provide sufficient funding so that each consortium has staff to meet the high 
demand for services. 

• Each consortium should develop an annual plan for services, fund-raising 
activities, staff development, and TLA topics, with consortia representation 
involved in the development of the plan. 

• The CTP at the state and regional levels should involve more administrators in 
planning and target some services to meet their needs at the school and district 
level. 

• The CTP should co-sponsor more activities with other education groups such as 
ACSA, the Subject Matter Professional Development Projects, the County State 
Steering Committee, and others. 

• Central CTP functions and special projects should be more systematically 
evaluated. 

• Continue to increase involvement of the CTP in areas that technology can 
support, such as school restructuring, alternative assessment, and learning 
resources management. 

• Consider adjusting the regional boundaries to decrease travel time, improve 
communication, and increase access to clients. 
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• Find ways to increase support from business and industry. 

• The CTP should be viewed as a vehicle for connecting the various agencies 
through telecommunications. 

• Increase the use of TRIE by making it easier to use and by providing more 
information of interest and use to a growing clientele. 

• Continuous evaluation of CTP activities at the regional and state levels should be 
conducted with CTP leadership using standardized forms and procedures to gather 
evaluation data and other information. 

• Establish clearly-stated outcome and implementation objectives for each of the 
programs. Such objectives should be the basis for program implementation and 
the evaluation of the overall impact of each program. 
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III. Analysis of Educational Technology Programs 
Funded From 1984-89 



The following analysis of programs funded under the provisions of AB 803 summarizes the 
strengths and weaknesses and makes recommendations about aspects of them that should be 
considered for future adaptation or replication. Detailed descriptions of these programs, from 
which this analysis is based, are provided in Phase I of the CETAP report. 

Several of the programs and projects described must be considered “risk ventures” that were bold 
efforts to meet the needs of teachers for technology resources. The period 1984 to 1989 was a 
time when educational technology was relatively new and changing at a rapid pace. 

A. Technology In the Curriculum (TIC) Guides 

The TIC project was funded during fiscal years 1985-86 and 1986-87 to match computer 
software and video programs with the curriculum, to identify subject and curriculum areas 
lacking in quality programs, and to prepare and distribute resource guides describing the 
available programs. This was a major initiative to evaluate and identify high quality computer 
and video programming. Projects in each major subject area determined which elements of the 
curriculum could be best taught with technology, identified high-quality programs that supported 
the curriculum frameworks, and prepared model lesson and unit plans that demonstrated how to 
integrate technology with the curriculum. The activities of the TIC project have been continued 
in part by the Software and Video Clearinghouses. 

Data Sources: The data sources for the following analysis included various teacher surveys, 

CTP surveys, CDE records, and staff comments. 

Strengths/Facilitating Factors 

• The TIC Project produced computer and video program resource guides that helped many 
teachers select and make effective use of high quality computer software and video 
programs as tools to augment curriculum and instruction. 

• The effectiveness of software and video programs was assessed by teams of experienced 
teachers so that other teachers would have a better idea of what programs would be 
effective. This reduced the likelihood that programs would be purchased, found 
inappropriate, and then not used. 

• The TIC materials provided a framework for the staff development programs that were 
conducted by the Technology Education and Computer (TEC) Centers and selected county 
offices of education. 

• The TIC project had the potential to benefit most schools in the state that used technology. 

• Each of the TIC projects identified curriculum areas that were lacking in high quality 
software. This information was used to determine curriculum areas to be addressed by 
state-funded software development projects. 

• TIC guides were distributed to every public school in California. 

Weaknesses/Constraints 

• Funding was not available to conduct a formal evaluation of the impact of the TIC project. 
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• According to a CTP survey, fewer than half of the schools that received the TIC Guides 
actually used them. 

• The print materials were too expensive to reach an adequate number of teachers. 

• The DataRelator program (a relational database for selecting TIC software) cost $100,000 
to develop but was found to have only a limited scope of information and was rarely used. 
This program was found to be useful by only one percent of the respondents to a survey 
conducted by the San Mateo County TEC Center. 

NOTE: The original DataRelator search concept has been modified and the search 
strategies originally set up for stand alone Apple computers are now in place using TRIE, 
the CSUNet electronic information service. 

Recommendations/Promising Practices 

• The TIC resources should be more effectively marketed. 

• The TIC guides should continue to be updated to accommodate new curriculum 
frameworks, combined with the ongoing reviews conducted by the Computer Software and 
Instructional Video Clearinghouses, and be made available to schools free on TRIE or at 
cost of production in printed versions. 

• TIC “exemplars” should be developed to accompany Program Quality Review (PQR) 
guidelines and self-study procedures. 

• County Office of Education staff, CTP Consortia, Subject Matter Professional 
Development Projects, and SB 1882 Staff Development Consortia should be involved in 
future distributions of technology support materials to increase infusion into education 
programs and initiatives. 

B. Technology in the Curriculum (TIC) Software Distribution 

The CDE purchased and distributed sample sets of exemplary software to all schools in the state 
along with the first set of TIC resource guides. The exemplary rated programs were included in 
the sample lesson plans in the TIC documents and were to serve as examples for training. 
Schools were required to send at least one teacher to a brief training session in order to receive 
the software. This was a one-time distribution, conducted in the Spring and Fall of 1986. 

Data Sources: The data sources for the following analysis included various teacher surveys, 
CTP surveys, and CDE records and staff comments. 

Strengths/Facilitating Factors 

• The state-wide purchase of computer software allowed substantial discounts to be 
negotiated with publishers. 

• The software and sample lesson plans in the TIC resource guides facilitated the integration 
of software and video with curriculum programs. 

• The distribution of software potentially benefited every school in the state; each school 
received TIC resource guides, sample software, and training. 

• The free samples of exemplary software encouraged schools to purchase additional copies 
with local funding, thereby promoting technology use. 

• Teachers who attended the brief training were able to go back to their schools and 
immediately put what they had learned into practice. 

so 
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• The software distribution provided an incentive for teachers to attend the TIC training 
sessions. 

Weaknesses/Constraints 

• There was no formal evaluation of the impact of the TIC software distribution project. 

• Since only one copy of each program was given to each school, teachers could not legally 
use the programs on more than one computer at a time. Long-term state-wide licensing 
would have increased access and use of the programs. 

• Due to a lack of funding, software was not distributed in conjunction with the Foreign 
Language and Visual and Performing Arts TIC guides. 

Recommendations/Promising Practices 

• Explore cost-effective ways to distribute technology-based materials. 

• State-wide purchases of exemplary technology materials can save a great deal of money for 
schools when compared to the prices paid by individual schools. 



C. Summer Technology Training Institutes 

During 1986 and 1987, the CDE funded nine four-week summer institutes for approximately 
1,200 teachers and other educators in the use of technology in the curriculum. Four elementary 
institute sessions focused on the integration of technology to support all subject areas while five 
secondary institutes gave attention to both integration of technology within specific subject areas 
and across curriculum areas. A “training of trainers” model was featured in the institute 
programs. Teachers were to become technology resources for their schools and/or districts after 
completing the institutes and were to provide local staff development and assistance. The 
summer institute program was not continued under AB 1470. However, the CTP regional 
consortia conduct Technology Leadership Academies which focus on the integration of 
technology into specific curriculum areas. 

Data Sources: The data sources for the following included a survey of past participants, CDE 
records, and comments by developers and implementers of the institutes. 

Strengths/Facilitating Factors 

• The Summer Institute Project helped to integrate the resources available from other state- 
supported projects, such as the Curriculum Implementation Centers (CICs), the TEC 
Centers, the Mentor Teacher Program, the TIC Projects, and the California Subject Matter 
Professional Development Projects. 

• The responses to a CETAP telephone survey of teachers who had attended the institutes 
were exceptionally positive; most stated that the information was highly useful and 
expressed interest in attending future institutes. 

• The use of universities as institute sites was reported to be both cost-effective and 
intellectually stimulating to the teachers and provided low-cost housing for participants. 

• Teacher response to the program was very positive, and most stated that the information 
received was highly useful. 
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Weaknesses/Constraints 

• The evaluations conducted by the directors of the nine institutes were inconsistent and 
lacked means to assess effects of institute training on student performance. 

• There was sometimes a lack of district support (funding for hardware, software, released 
days, etc.) for the teams of teachers to disseminate information after returning from the 
institutes. 

• Because of abrupt funding cuts and the elimination of the TEC Centers, little follow-up was 
conducted to determine how well teachers had disseminated the information received from 
the institutes at their own schools and districts. 

• In many cases the teachers were not able to work as teams to provide staff development 
after the institute, as had been planned. 

• Because of funding limitations, summer commitments, and limits on the numbers who 
could attend as an LEA team, only a limited number of teachers were able to attend the 
institutes. 



Recommendations/Promising Practices 

• Continue to offer the Technology Leadership Academies (TLA). 

• In-depth technology infusion institutes should be conducted in the future. 

• Continue technology use institutes co-sponsored by selected MTS projects, the CTP, ITV 
agencies, and the Subject Matter Professional Development Projects. 

• Assess the effectiveness of the TLAs now being conducted by the CTP. 

• Evaluate the effectiveness of future institutes including follow-up on the training of trainers 
component (if included). 



D. California Instructional Video Clearinghouse and 
California Computer Software Clearinghouse 

The California Instructional Video Clearinghouse was established in 1986 as a state-wide service 
to coordinate the evaluation of instructional video programs and to disseminate information 
about high-quality programming throughout the state. It has been administered by the Stanislaus 
County Office of Education (COE) in cooperation with other COE’s since its inception. 

The Computer Software Clearinghouse evolved from the TEC Center Software Library at the 
San Mateo County Office of Education. The Clearinghouse is responsible for coordinating the 
evaluation of educational computer software and disseminating information about high-quality 
programming throughout the state. It is currently located at the California State University, Long 
Beach. 

Both Clearinghouses were funded to update and expand the work of the TIC Projects in 
organizing information about technology-based curriculum resources that would serve as tools 
for teaching curriculum subject matter. They now collaborate in the evaluation of multi-media 
programs that involve both computer and video technologies. 
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Currently, up-to-date information on effective computer and video programs is available in the 
TRIE electronic database that can be accessed through the state-wide CSUNet. The Software 
Clearinghouse represents California in the national Educational Software Preview Guide project. 

Data Sources: The data sources for the following analysis included interviews of clearinghouse 
directors and CDE staff, CDE records, and articles about the clearinghouses. 



Strengths/Facilitating Factors 

• The clearinghouses are the major resource that evaluates new programs and re-evaluate old 
programs to reflect the state’s curriculum frameworks. 

• The clearinghouses coordinate activities to ensure uniformity of evaluation guidelines and 
to avoid duplication of effort. 

• Those who use the clearinghouse resources report that time and effort are saved in 
searching for exemplary programs. 

• Extensive partnerships with publishers have supported the goals of the clearinghouses 
through the donation/loan of video programs. 

• The clearinghouses are jointly endorsed by the County State Steering Committee. 

• All counties can use the reviews without duplicating this effort. 

Weaknesses/Constraints 

• The “visibility” of the clearinghouse services among classroom teachers is low. The TRIE 
database and the publications of the software and video clearinghouses seldom come to the 
attention of many educators other than instructional media specialists. 

• The level, type, and frequency of use of the clearinghouse services has not been assessed. 

Recommendations/Promising Practices 

• More detailed information on classroom applications and the curriculum content of 
programs is needed in the TRIE database entries. 

• TRIE needs to be more “user friendly” and additional local access numbers (or a state-wide 
toll-free 800 number) are needed. 

• The Guidelines for Computer Software and Instructional Video should continue to be 
periodically revised to keep up with changes in technology and curriculum standards. 

• A state-wide survey of the level and type of use by educators of the Computer Software and 
Instructional Video Clearinghouses needs to be conducted. 

• The CDE should continue to provide funding to operate the California Instructional Video 
and Computer Software Clearinghouses. 

• Application tools, such as word processing software and spreadsheets, should be reviewed 
in addition to stand-alone educational software, and examples for integrating applications 
with the curriculum should be provided. 



E. Teaching Video Pilot Program 

The Teaching Video Pilot Program (TVPP) was established by special legislation, Senate Bill 
2130 (Seymore), to fund a project to determine the effectiveness of teacher-produced 
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instructional videos for classroom use. During 1987, the successful bidder on the TVPP project, 
the Sacramento Educational Cable Consortium, (SECC) produced 125 videotapes and 
broadcasted them over the local cable system. Thirty-four high school teachers in six districts 
were trained in video production. 

Data Sources: The data sources for the following analysis included the TVPP Final Evaluation 
Report and CDE records. 

Strengths/Facilitating Factors 

• The TVPP project demonstrated clearly that — given proper training and equipment — 
teachers could successfully produce high-quality instructional programming to support 
classroom instruction. 

• The 125 programs were broadcast over the local cable system. 

• Students were actively involved in the production of the TVPP video programs. 

• A variety of local businesses, foundations, and institutions of higher education contributed 
to the TVPP project. 

• The TVPP teachers continued to develop other programs after the project funding 
terminated. 

Weaknesses/Constraints 

• Teachers needed more time than was available to produce the video tapes. 

• District support for the project was inconsistent among the six participating districts - some 
provided teacher released time, and some did not. 

Recommendations/Promising Practices 

• The CDE should use the various regional support programs (Subject Matter Professional 
Development Projects, SB 1882 Staff Development Consortia, the CTP Technology 
Leadership Academies) to disseminate information about promising practices developed by 
the TVPP. 

• In any future project of this type, sufficient released time for the teachers should be 
provided by participating school districts. 



F. California Historical Society On Location Video Programs 

In 1987, the CDE awarded a grant to the California Historical Society (CHS) to produce a series 
of ITV programs with teacher’s guides designed to support the fourth grade history-social 
science curriculum. This project was viewed as a potential long-term developmental partnership 
that would help to increase student interest and knowledge in California history. Four programs 
were produced and are currently available to educators. 

Data Sources: The data sources for the following analysis included CDE records and staff 
comments and evaluation reports for some series. 
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Strengths/Facilitating Factors 

• The programs produced reflect the cultural diversity of California and covered topics which 
are not emphasized or even covered in textbooks. 

• The CHS raised significant funding (over $1 13,000) to help support production of the 
series. 

• Extensive curriculum materials were developed to accompany the television programs. 

Weaknesses/Constraints 

• The CHS encountered serious financial problems shortly after production of the On 
Location series began. This contributed to the decision to withdraw from the project after 
having produced three of the eight programs. 

• Two of the original three videos produced were found to be of poor quality by many 
educators, and none are listed among the popular ITV titles. 

Recommendations/Promising Practices 

• Continue to explore development of high-quality technology-based materials to supplement 
the California history curriculum. 

• Involve the appropriate curriculum office of the CDE, the Subject Matter Professional 
Development Projects, and a production advisory committee in the development of any 
future programs. 



G. VCR Distribution 

In September of 1985, the CDE distributed a videocassette recorder to each public school in 
California. To receive the VCRs, schools were required to send a representative to a short staff 
development session covering basic VCR operation, curriculum integration, ITV resources, and 
copyright law. There have been no other state-wide distributions of equipment. 

Data Sources: The data sources for the following analysis included CDE records and staff 
comments. 



Strengths/Facilitating Factors 

• According to anecdotal reports, the program stimulated increased use of instructional video 
in classrooms.- 

• Approximately $1.5 million was saved in comparison to what the schools would have paid 
individually for comparable VCRs. 

• The staff development and distribution services of the county offices of education and ITV 
agencies were contributed at no additional cost to AB 803. 

• The VCR distribution provided an incentive for teachers to attend training sessions on the 
use of video to improve classroom instruction. 

• The program provided teachers with the ability to record programs to be viewed at a later 
date. 
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Weaknesses/Constraints 

• A follow-up evaluation of school use of the VCRs was not conducted. 

• The allocation of one VCR per school was not equitable because teacher access to the 
equipment at small schools was higher than at large schools where the VCR would have to 
be shared by more teachers. 

Recommendations/Promising Practices 

• In any future programs of this type, instructional video programs, appropriate for different 
grade levels, should be included with the equipment, along with sample lesson plans and 
activities so that teachers can begin using what they learn immediately. 

• To ensure equity in future distributions, school size should be taken into account. 

H. ITV Licensing 

In 1982, the CDE established a program of acquiring multi-year licenses from the producers or 
distributors of ITV series for the rights to broadcast and distribute the series to schools in the 
state. A committee of the California Instructional Video Consortium (CIVC), made up of the 
directors of the seven ITV agencies, with input from county media directors and CDE subject 
area consultants, reviews and makes recommendations to the Educational Technology 
Committee for state- wide licensing of ITV programs. For the past ten years, the state has funded 
licensing at approximately $250,000 per year. ITV licensing is an ongoing program that is 
currently funded by AB 1470. 

Data Sources: The data sources for the following analysis included CDE records, CDE staff 
comments, and comments by ITV agency staff members. 

Strengths/Facilitating Factors 

• Many thousands of dollars in savings are achieved over the amount that the ITV agencies 
or school districts would have to pay individually to license high-quality video 
programming regionally or locally. 

• Programs are reviewed by teachers and ITV regional agencies for alignment with the 
California curriculum frameworks. 

• State-wide licensing promotes equitable access to ITV programming across the state. 

Weaknesses/Constraints 

• Concerns have been expressed by the CDE about the degree to which the programs are 
actually aligned with the curriculum frameworks. 

• The ITV licensing program is not formally coordinated with the evaluation work of the 
California Instructional Video Clearinghouse. 

Recommendations/Promising Practices 

• Student input should be continually utilized in choosing series for licensing. 

• Sample surveys of teachers should be conducted periodically to determine the level of use 
and the effectiveness of the currently licensed ITV series. 
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• The California Instructional Video Clearinghouse should have a formal role in the ITV 
licensing and selection process. 



I. Educational Telecommunications Network (ETN) Staff 
Development Program 

In 1989, the CDE provided funding to the Educational Telecommunications Network (ETN) at 
the Los Angeles County Office of Education to produce a series of programs on using 
technology to support instruction. The programs were broadcast early in 1990 over the ETN 
satellite network. ETN continues (with local and membership funding) and has greatly expanded 
its role as a vehicle to deliver a wide variety of programs designed to increase the knowledge and 
skills of educators across the state. 

Data Sources : The data sources for the following analysis included comments by present ETN 
staff and brief reports. 

Strengths/Facilitating Factors 

• The program utilized telecommunications to deliver staff development as a means of 
increasing access at a lower cost. 

• Though not funded by the state, ETN is often cited as an effective staff development . 
delivery system. 

• Production of the video segments illustrating classroom integration of technologies 
involved expert practitioners and subject matter instructional consultants. 

Weaknesses/Constraints 

• The small size of the grant award limited the quality and depth of the production. 

• Evaluation data were not collected. 

• ETN has not disseminated information on the programs and has no information on the use 
of the programs. 

• The number of educators who benefited from the series is unknown (this is a common 
problem with distance learning services). 

Recommendations/Promising Practices 

• Continue to explore the use of ETN facilities and services to provide distance teaching, 
and teacher training. 

• Consider funding specific uses of ETN or other distance learning to increase access of 
information to rural parts of California. 



J. California Mechanical Universe Model 

The California Mechanical Universe Model was a collaborative project developed and 
implemented by Caltech, the California State University (CSU), and the CDE. The intent of the 
project was to provide inexperienced high school physics teachers with training in the use of the 
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Mechanical Universe video series and other demonstration aids. These two week trainings were 
conducted during the summer of 1989 at five CSU campuses. 

Data Sources: The data sources for the following analysis included CDE records and project 
evaluation reports. 

Strengths/Facilitating Factors 

• The Mechanical Universe videotapes and materials were provided to the project by Caltech 
at a greatly reduced cost. Many of the demonstration aids were also procured at greatly 
reduced prices. 

• This project reportedly met many of the needs of inexperienced high school science 
teachers and those who were not certified to teach physics. 

• The CSU campuses that conducted formal evaluations of the institutes received 

overwhelmingly favorable responses from the teachers, with most expressing an interest in 
attending future summer institutes. ’ 

• Extensive information was developed and provided for the integration of the Mechanical 
Universe series with the high school physics curriculum for more than 100 physics teachers 
(one fifth of the state’s schools). 

Weaknesses/Constra ints 

• The organizers of the institutes reported that the $15,000 grants were much too small; some 
were unable to provide teachers with extensive demonstration equipment, and others had to 
charge extra for it. 

• Long term outcomes of the institutes, particularly the use of the knowledge and materials 
provided by teachers in classroom instruction, were not evaluated. 

Recommendations/Promising Practices 

• Staff development opportunities related to the science instructional video series and a 
program of extensive follow-up activities should be provided for teachers who are 
inexperienced or not credentialled in the subject they are teaching. 



K. Developmental Grants Program 

A series of developmental projects were funded by the CDE between 1984 and 1987 to develop 
and validate models for technology use that could be disseminated to other schools in the state. 
Several projects competitively funded for about two years and then abruptly terminated due to 
cuts of state funds. Projects that developed effective products and practices could later apply for 
dissemination grants. 

Data Sources: The data sources for the following analysis included CDE records, project 
descriptions, and comments from project developers. 

Strengths/Facilitating Factors 

• Schools were provided with the opportunity and incentive to develop new and innovative 
strategies for integrating technology with the curriculum.* Repons from developers 
indicate that potential promising practices and products may have resulted from these 
projects 
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Weaknesses/Constraints 

• Due to budget cuts, the AB 803 developmental projects were discontinued before they 
could be completed and validated. 

• Because projects were not completed, the promising products or practices resulting from 
the developmental grant projects funded under AB 803 were not fully identified, validated, 
or funded for dissemination. 

Recommendations/Promising Practices 

• A developmental (or R&D) grants program should be considered to establish and validate 
new model programs. 

• The forms and procedures applied to the developmental grants should be considered for 
adaptation to future developmental or R & D grants programs. 



L. Dissemination Grants Program 

Dissemination grants were awarded to several projects (which had been funded under the 
educational technology legislation in effect before AB 803) that had created effective models for 
technology applications. These projects, and, subsequently, those funded by AB 803, were to 
establish adopters and provide training on a state-wide basis. The dissemination program funded 
27 projects in 1983-84 and 1984-85, 11 in 1985-86, and 10 in 1986-87. Funding was terminated 
in July 1987 when the state Educational Technology program budget was reduced 50 percent by 
the Governor. Projects prepared and distributed materials for students and teachers, hosted 
visitations, made presentations, trained staff at the regional TEC centers, conducted follow-up 
technical assistance when possible, and occasionally evaluated the adoption of their practices. 

Data Sources: The data sources for the following analysis included CDE records, project 
surveys and reports, and comments from project developers. 

Strengths/Facilitating Factors 

• The program capitalized on the strengths of successful projects by disseminating effective 
products and practices of these projects throughout the state. 

• Collaboration with the TEC Centers provided a cost-effective means for dissemination of 
the model programs and materials. 

• Awareness presentations were conducted to educators representing more than a quarter of 
the schools in California. 

• Thousands of educators at hundreds of schools were reported to have been trained by these 
projects. 

• Over 30% of the AB 803 Adoption/Expansion Grant projects were reported to be 
adaptations of the Dissemination projects. 

• Presentations and showcasing of the programs were conducted at conferences. 
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Weaknesses/Constraints 

• Due to budget cuts, the AB 803 projects were discontinued before their state-wide impact 
could be assessed. 

• The dissemination program was not well coordinated with existing staff development and 
regional service agencies. 

Recommendations/Promising Practices 

• Incentives should be considered for the “packaging” and dissemination of the innovative 
practices developed by the current School-Based Educational Technology Projects. 

• A state-wide clearinghouse and database of “successful” technology-based material and 
practices should be established and maintained in collaboration with the Subject Matter 
Professional Development Projects, county offices of education, CTP, and ITV agencies. 

• A dissemination grants program (beyond the current Level II Model Technology School 
program) should be reestablished to enable schools to adopt and/or adapt successful models 
of innovative strategies for integrating technology with the curriculum. 



M. Adoption/Expansion Grant Program 

The AB 803 Adoption/Expansion Grant program provided small grants to schools for the 
development of projects including the purchase of hardware and software and integration of 
educational technology resources. The program was also established to serve as a catalyst to 
promote greater use of local resources and a commitment to sound planning. Schools were 
encouraged to plan for curriculum integration and to provide appropriate staff development. The 
TEC Centers provided local administration, staff development, and technical assistance to the 
grant developers and recipients. This program operated from 1984-1989 and provided funds to 
5,638 school sites. 

Data Sources: The data sources for the following analysis included previous studies, the CDE 
Sunset Review, CDE staff comments, and input from program implementers. 



Strengths/Facilitating Factors 

• The adoption/expansion grants appeared to have great potential impact on students and 
teachers in that over 70 percent of the schools in the state received Adoption/Expansion 
Grants, 

• District level technology use planning involving parents, teachers, administrators, regional 
organizations, and business/industry was encouraged and initiated in many districts. 

• School planning, though not formalized, was expected to be consistent with the 
comprehensive technology and/or computer plans in the districts. 

• Rating criteria for judging proposals emphasized careful planning and coordination to meet 
student and teacher needs for staff development. 

• Program guidelines went beyond legislative requirements to require grant recipients to 
allocate at least ten percent of project funds to staff development. 

• According to the CDE’s Sunset Review, grant awards were equitably distributed. 
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• Teacher and student access to technology was significantly improved throughout the state; 
the number of computers in schools that received grants increased by 40 percent. 

• Schools and districts provided additional resources well beyond the required ten percent 
match. 

Weaknesses/Constraints 

• There was no funding available to conduct extensive follow-up on the project evaluations 
required by the CDE; some projects prepared reports, but they were never collected and 
reviewed. 

• The elimination of the TEC Centers left the the last phase of Adoption/Expansion projects 
without implementation assistance and regional staff development. 

• It was reported that the district technology plans usually were not taken seriously and 
implemented. 

• The lack of coordinated school-level planning sometimes resulted in isolated uses of 
technology in particular classrooms. 

• Some projects did not incorporate the use of existing technologies and other school and 
district resources. 

Recommendations/Promising Practices 

• The most promising aspects of the AB 803 Adoption/Expansion grants program, such as 
the staff development requirement, were incorporated into the AB 1470 School-Based 
Educational Technology Grants Program. 

• Most of the deficiencies in the AB 803 program were considered in the formulation of AB 
1470 which included emphasis on school-level planning for technology use, emphasis on 
implementing ongoing district and school planning, and optimal use of learning resources. 

• Establish a state-wide clearinghouse for information and research, resources, practices, and 
technology-based materials to support the developers of school-level projects. 

• Encourage projects to adapt existing promising and exemplary models for technology use. 
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IV. Summary, Conclusions ,and Recommendations 



This section describes the costs for each of the six projects and programs studied in Phase II. 
Included is a brief analysis of the programs in relation to the implementation of state initiatives. 
Finally, a set of recommendations that emerged from the analysis of the study are suggested. 



A. Summary of Cost Across Projects 

(Section IV-A prepared by the American Institutes of Research) 

The primary cost-effectiveness question the state must ask in relation to the six technology 
projects currently being funded under AB 1470 is whether the results of these programs have 
justified the expenditures to date, and whether the state should maintain funding at the current 
level, continue support at a diminished or enhanced level, or discontinue the programs. 

This type of evaluation has been hindered for these projects by the lack of clearly stated common 
goals and objectives across the projects, or even within projects. For example, to compare the 
relative cost-effectiveness of the School-Based Grant program as compared with ITV, we would 
want to compare how effective they had been in realizing certain common goals and objectives 
per unit of cost. Even within a single program (e.g., Level I Model Technology Schools), the 
individual project sites are sufficiently unique and the project goals too diverse to make 
meaningful comparisons difficult. 

What can be done, however, is to develop a common set of questions related to program costs 
and effects and to consider each program individually on the basis of these common criteria. 

First, to what extent was there sufficient interest in the program to generate supplemental support 
from the host agency and other entities to "leverage" the state investment? Second, what appear 
to be the primary purposes of the program from a state perspective, and what indicators show the 
relative success of the program in achieving them? 

School-Based Grants : For every state dollar invested in School-Based Grants, the grant 
recipients report matching funds at a rate of eighty cents. The major purpose of the School- 
Based Grant project is to provide seed money to foster the use of technology and to integrate it 
into the curriculum at individual school sites. With total funding of $5,999,285 for the 1990-91 
school year allocated to 320 schools, the average amount of AB 1470 funds received for this year 
was $ 1 8,748 per school. The criterion for evaluating the project was the successful 
implementation of technology at each of these school sites. Effectiveness indicators are: (1) a 
sample of teachers rating the benefit of these projects to their school as significant; and (2) 
moderate to significant gains in teacher proficiency and student outcome indicators. 

Level I Model Technology Schools: For every state dollar invested in this program, grant 
recipients reported other funds invested in technology at the project sites of $1.17 for the 
1990/9 1 year. The major purpose of the MTS Level I project from the state perspective is to 
provide models of school-wide technology use that impact student learning and that the other 
schools of the state can visit, learn from, and emulate. A second objective is the production and 
dissemination of technology-related projects and services. Third, important research goals are 
associated with each of these projects, with approximately 15-20% of each project budget being 
allocated to research. With total funding of $3,100,000 for the 1990-91 school year allocated to 
17 schools across six projects, the average amount of AB 1470 funds received for this year was 
$182,353 per school. Effectiveness indicators are over 5,000 visitors reported across the six 
projects, as well as 576 other dissemination activities since the onset of these projects. 
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Level II Model Technology Schools : For every state dollar invested in this program, grant 
recipients reported matching funds of 61 cents for the 1990/91 year. In contrast to the three sets 
of goals stated for the MTS Level I projects, the single major objective of the MTS Level II 
project is to integrate technology into a single curriculum content area and to produce model 
products and practices for other schools to adopt or adapt. With total funding of $961,266 for the 
1990-91 school year allocated to six schools, average amount of AB 1470 funds received for this 
year was $160,21 1. An effectiveness indicator is the 172 schools reporting adopting or adapting 
Level II projects. 

Instructional Television Agencies: For every state dollar invested in this program, grant 
recipients reported matching funds and fees of $2.33 for the 1990/91 year. The primary 
objective of the ITV project is to provide television programming designed to enhance 
instruction. With total funding of $1,908,791 for the 1990-91 school year allocated to seven 
agencies, average amount of AB 1470 funds received for this year was $272,684. An 
effectiveness indicator is that approximately 60% of the principals responding to the state-wide 
CPB survey reported that instructional television had a moderate to considerable affect on 
student learning and motivation, and on teacher effectiveness. 

California Technology Project: For every state dollar invested in this program, an additional 
dollar in matching funds was reported for the 1990/91 year. The primary objective of the CTP is 
to meet a variety of state-wide needs related to the implementation and coordination of education 
technology resources in schools. Total AB 1470'funds received for this year for the 1990-91 
school year was $1,095,500. An effectiveness indicator for this project is the over 60% of 
districts responding to a survey for this project rating the CTP as a cost-effective vehicle for 
delivering information and staff development resources to schools. 

The Software Development Project: The primary objective of the Software Development Project 
is to provide "seed money" to encourage publishers to contribute investment capital in 
educational software designed to align with the needs of the California Curriculum Frameworks. 
Effectiveness indicators are that two of the grant recipients report that they have had sufficient 
success in marketing their products to recover their investments to date in the project, and the 
state has received royalties on out-of-state sales. 

In summary, the projects were not structured in a way that allows a comparison of their cost- 
benefits in relation to one another. Such comparisons require that common outcome measures be 
maintained across projects. If it is not feasible to specify common objectives at the outset of such 
projects because the goals of the individual project components are diverse; the next best 
alternative would be to specify criteria internal to each project upon which the relative 
effectiveness of each project component will be measured. Data on the results of each project on 
these criteria can then be gathered at regular intervals (e.g. annually), to compare the relative 
cost-effectiveness ofithe sites within each project and to subjectively judge the outcomes to date 
for the project as a whole in relation to its cost. In the case of the current evaluation, due to the 
absence of such predetermined and clearly defined outcome measures, the preceding section has 
reported the overall cost of each project, the cost per site, and the most objective measures of 
effectiveness that are available retrospectively for each project component. 

B. Overall Analysis 

Since 1984 the major goal of the state’s educational technology program has been to enhance the 
classroom use of the state curriculum frameworks with the aid of technology. The CDE, with the 
advice of the Educational Technology Committee, and with input from educators, have 
conceptualized and implemented over 25 programs and projects, each with a direct or indirect 
emphasis on curriculum implementation. To further this aim, the CDE and the Legislature have 
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strongly emphasized program elements to include school and district technology use planning, 
staff development for effective implementation, effective management of educational resources, 
and evaluation of the benefits of technology-based curriculum programs. The following analysis 
summarizes the relative influence that the six programs exerted on the state’s educational 
initiatives. 

School-Based Educational Technology: The School-Based Educational Technology (SBET) 
projects played a major role in stimulating the site-based planning for technology use as was 
emphasized by the Legislature and the CDE. Along with this planning was the incorporation of 
technology into curriculum and instruction. Staff development was also emphasized and 
evaluation was implemented in most projects. The SBET projects evidenced much promise in 
producing technology use planning and curriculum integration. However, more project 
implementation time is needed to adequately assess impact on teaching and learning. The 
CETAP study clearly shows that the guidelines combined with the funding incentives and 
interests stimulated by technology, influenced implementation of the state’s initiatives for SBET 
projects, especially site planning and curricular integration of technology. The influence of the 
SBET projects would have been minimal or nonexistent without the support of the CDE Office 
of Educational Technology ensuring that the guidelines were well defined and put into operation 
by regional agencies such as the CTP. 

California Technology Project: The CTP carried out its initial implementation of initiatives by 
linking the state to the local projects with developmental training that included technology use 
planning, curriculum integration through Teacher Leadership Academies (TLAs) and evaluation 
training. The CTP had a major influence on informing educators about ways to implement 
technology resources, especially site planning, curricular integration workshops, and technical 
assistance. The CTP provides follow-up assistance to projects for implementation and evaluation 
while linking them to the Technology in the Curriculum resources listed in the TRIE database 
and featured in the TLAs. 

Instructional Television: The ITV agencies indirectly support the CDE initiatives by assisting 
schools in the selection of ITV programming that supports the frameworks and by providing 
print materials to guide teachers in the integration of video programs into the curriculum. ITV 
agencies support development and implementation of SBET project initiatives, especially 
professional development of teachers and curriculum support. The ITV agencies also 
implemented their own plans that reflect the CDE initiatives. 

Level I and II Model Technology Schools: The Level II MTS projects provide support for the 
targeted implementation of frameworks by directly assisting schools to adopt specific 
framework-based technology applications in specific subject areas. The MTS Level II projects 
have the greatest impact on school improvement, staff development, and curriculum and 
instructional support. 

Level I MTS projects emphasize school and district planning models by showing schools 
methods to manage and direct learning resources that support the planned integration of 
technology. These projects have produced and field-tested promising practices and products that 
can be funded through future grants, contracts, or service fees to help other schools plan, 
implement, and evaluate technology. 

Software Development Projects: These projects are the most specialized in their focus on 
curriculum and provide schools with technology-based materials to augment or even replace 
existing print materials and approaches for developing framework-based objectives for students. 

In general, the programs funded by AB 1470 all have varied approaches to interactively support 
the increased planned use of technology to augment and support the state curriculum 
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frameworks. The results of CETAP indicate that while all programs support the state’s 
instructional improvement objectives, much can be done to improve this effort. The many 
recommendations in this report could help to increase the collective and individual impact of the 
state’s educational technology initiatives. 



The following chart shows the emphasis of each program on initiatives as determined by the data 
and information collected in the study. These initiatives are the guiding principles for 
technology use. 



Educational Technology Projects and Their Relative Level of 
Impact on Major State Initiatives Directed Towards School Sites 
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C. Recommendations 

The following recommendations are based on the analysis of the data and information collected 

directly from the stakeholders, project records, and the many other sources cited in this study: 

1. School-Based. Projects: The planned use of a variety of technologies to enhance and expand 
the state curriculum frameworks should be continued. Incentives will continue to be needed 
to initiate technology planning and use in schools that have not benefited from the funding 
made possible by AB 803 and AB 1470. 

2. Technology Use Planning: Planning for technology use is critical and should continue to be a 
priority with increased emphasis at all levels from the classroom to the district level. New 
programs should continue the emphasis on adapting existing school plans to include 
technology use and increase emphasis on coordinating school planning with district planning. 

3. Staff Development: Continue to emphasize school-based staff development that ensures 
individual teachers will be able to acquire the knowledge and skills they need to utilize new 
technology-based learning resources. At least 10% of project budgets should continue to be 
mandated for staff development in any school level program. 
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4. Regional Support: Improve coordination and delivery of state and regional support to schools 
for technology use. Consider restructuring, consolidating, and better coordinating some 
aspects of the regional support systems such as the CTP, ITV, CDE initiatives and programs, 
and institutions of higher education. Review all available regional school support services 
for K- 12 and post- secondary to consider ways to interconnect agencies and optimize their 
efficiency in the support of technology use in all schools. 

5. Statewide Database: Establish a state-wide centralized clearinghouse with a database of a 
large pool of promising practices, curriculum products, and model programs. Include a 
process to identify, describe and disseminate staff development opportunities, promising 
technology applications, technology-based materials, and support services to educators 
throughout California. 

6. Resource Development and Dissemination: Provide incentives and opportunities for 
educators to develop, validate, and share promising products and practices for applying new 
and emerging technologies that improve instruction for students. Establish and implement 
more clearly defined standards for development, evaluation, and dissemination of programs- 
Funding priorities for these programs should be guided by student and teacher needs as well 
as emerging state and national educational trends. 

7. Use of Current Models: Ensure that effective aspects of programs and projects already in 
place are carefully considered in developing new initiatives and programs. Considerable state 
and school district investment in models has already occurred. For example, the MTS Level 
I and II projects, as well as many school-based educational technology grants, have produced 
many products and practices that need to be shared and built upon. 

8. Telecommunication and Distance Learning: Expand and modify the current 
telecommunications systems to become a coordinated, user friendly, and cost effective 
network that provides timely and relevant information for use by educators and other 
education stakeholders. The system should allow for easy exchange of information within 
and between all education agencies. 

9. Instructional Video: Establish financial incentives for the development of effective 
educational video programs for students, staff development programs for teachers and 
administrators, and increased use of interactive teleconferences. Programs should align with 
the California Curriculum Frameworks. 

10. Business Partnerships: Continue efforts to establish project-focused collaboration between 
business and education. Retain the current software development partnership project model 
to include involvement of schools/districts as partners with business in the development of 
broad technologyJ)ased approaches. 

1 1. Assessment: Continue the emphasis on assessment of program and project development, 
dissemination and impact with special emphasis on making assessment and evaluation design 
part of initial planning and development. The CDE should develop and implement clearly 
defined standards and expectations with proposed instrumentation for all programs and 
projects. The CDE should establish guidelines to use “alternative assessment” 
methodologies appropriate for project and program evaluation and to suggest ways that 
technology can be used as a tool for implementing assessments. 

12. Use of Assessment Findings: The CDE should continuously collect and report findings 
about desired practices and products that emerge from the state supported programs. 
Evaluation information from this and other studies such as the MTS Research and Evaluation 
should be used to help guide the CDE in program planning and development decisions. 
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13. Inter-agency and inter-project coordination: The technology projects assessed in the 
CETAP study have been held accountable by the CDE for ensuring that their focus is the 
integration of technology into the curriculum. In order to fully realize this goal, other state 
projects, such as the Subject Matter Professional Development Projects should be required to 
incorporate the effective use of technology. Similarly, other offices, divisions, and branches 
of the CDE should address the goal of technology integration. 

14. Targeted student populations: Establish incentive grants matched to other categorical 
programs for targeted student populations to encourage technology applications to 
educational solutions. Targeted programs/populations include special education, alternative 
education, compensatory education and ESL/bilingual programs. 

15. Quality Indicators: The program quality indicators used for the School Improvement 
Program should be revised to encourage technology integration in the schools. New PQR 
guidelines should include “exemplars” for the effective use of technology to enhance and 
improve instruction. 

16. Funding: The Legislature should ensure that sufficient and sustained funding is allocated to 
fully implement these and other recommendations selected for implementation This includes 
funding for districts to retrofit existing facilities to support technology. 
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Appendix A. Cost-Benefits 



The lack of common evaluation criteria over the grant sites and programs means that 
they cannot be evaluated using a true cost-benefit framework. Such analyses are 
always comparative in nature. We can, however, express certain outcome indicators for 
each program and project as a whole in relation to program costs. 

A. School-Based Grants 

• The in-kind and direct cash support reported by the projects in support of the AB 
1470 grants added an additional 80.3% toward the support of these projects. (1) 

• Most of these matching funds came from other district funding sources, e.g. 

29.5% came from district general funds. Nearly 25%, however, came from 
sources external to the districts, e.g. community and business donations. (1) 

• Matching funds for these programs substantially exceeded what was originally 
proposed. Actual direct matching funds exceeded what had been proposed by 
37.7%, and in-kind resources exceeded what had been proposed by 45.7%. (1) 

• Nearly 80% of project funds were expended on technology related equipment, 
supplies, and materials. (1) 

• On a scale of 1 to 5, with 5 representing significant gain, teachers responding to 
the teacher survey rated the benefit of the project to the school as a whole as 4.5. 

(3) 

• Teachers report moderate to significant gains in a broad range of teacher 
proficiency indicators as a result of the project. (3) 

• Teachers report moderate to significant gains in a broad range of student outcome 
indicators as a result of the project. (3) 

• Students report "a little" to "some" gains in selected attitudinal and outcome 
variables as a result of the project. (4) 

B. California Technology Project Regional Consortia 

' • More that 50% of the revenues for the projects for the 1990/91 year came from 
sources other than AB 1470 grants. Major other sources of revenue included 
workshop fees (16.3%), membership fees (19.5%), and direct fiscal agency 
contributions (17.6%). (1) 

• Substantial in-kind support was received by the Consortia during 1990/91. Over 
$350,000 was received in in-kind support for this year, which matches 73.4% of 
total project revenues, and represents 168.2% of the total AB 1470 grant amount. 

(1) 

• Including the total direct revenue and in-kind assistance for the Consortia in 
1990/91, the AB 1470 grant only represented 25.2% of total support. (1) 

• Of the 31 1 school districts responding to the Assessment of CTP Services 
administered through this evaluation, approximately 35% to 60% responded that 
they used the services provided by the CTP. (2) 
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• Over one-half of the district respondents had a school apply for a School-Based 
Technology Grant, and 86.1% of those who applied attended a CTP-sponsored 
workshop or received assistance from the CTP during this process. The value of 
the services provided were rated from moderate to high. (2) 

• The majority of the districts (60.8%) responded that they found the CTP to 
provide a cost-effective vehicle for delivering information and staff development 
resources to schools. Most of the other respondents (32.6%) said they did not 
have enough information to respond to this question. (2) 

C. Instructional Television Regional Agencies 

• Nearly 70% of the revenues for the ITV Agencies for the 1990/91 year came from 
sources other than AB 1470 grants. Major other sources of revenue included 
direct fiscal agency contributions (49.2%) and membership fees (12.9%) (1) 

• Over $240,000 was received in in-kind support for during 1990/91. This matches 
17.2% of total project revenues, and represents 68.4% of the total AB 1470 grant 
amount. (1) 

• Including the total direct revenue and in-kind assistance for the ITV Agencies in 
1990/91, the AB 1470 grant only represented 25.2% of total support. (1) 

• Over 65% of the superintendents responding to the specific California items on 
the CPB Survey indicated that they were somewhat (37.3%) to very (27.8%) 
satisfied with the ITV Agency services provided them. (3) 

• Of the principals responding to the specific California items on the CPB Survey, 
nearly 60% or more responded that instructional television has a moderate to 
considerable effect on student learning (64.1%), student motivation (68.5%), and 
teacher effectiveness (59.9%). (3) 

D. Model Technology Schools Level I 



• The number of students directly or indirectly served by the project varies from 1 1,093 in 
the Los Angeles project to 834 in the Hueneme project. (1) 

• This results in an AB 1470 grant that ranges from $45 per student per year in the Los 
Angeles project to $600 per in the Hueneme project. (1) 

• Including the full technology expenditures from all sources at the project schools reported 
by the project sites, the total expenditure on technology across the project schools ranges 
from a little more than $86 per student in Los Angeles project to $2,780 per student in the 
Hueneme project. These are the extremes, with the other four project sites ranging 
between $156 and $312 per student. (1) 

• The technology donations received by the project sites are at least partly the result of the 
technology award from the state. Comparing the donations received during the 1989/90 
school year by each site in relation to the $500,000 allocation received by the state, it 
could be argued that every dollar allocated by the state "leveraged" supplemental support 
for these technology projects in the form of donations. Across all projects, the average 
amount per state dollar leveraged through donations was $0.27. (1) 

• Counting district support and donations project-wide, every state dollar equalled $1.17 in 
other technology expenditures. (1) 
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Appendix A. Cost-Benefits 



• In addition to the number of students enrolled at project schools, three other outcome 
measures are informative in considering the overall benefit of these projects to the state. 
These are the total number of active participants in the project, the total number of 
visitors to the project, and the total number of dissemination activities in which the 
projects have been engaged. 



E. Model Technology Schools Level II 

• More than 42% of the revenues for the Level II MTS Project for the 1990/91 year 
came from sources other than AB 1470 grants. Other major sources of revenue 
included direct fiscal agency contributions (7.4%) and in kind contributions 
(25.3%) (1) 

• Students report modest gains in selected attitudinal and educational outcome 
variables as a result of the project. (2) 

• A total of 172 schools reported adopting or adapting Level II MTS projects. Of 
the School-Based Grant projects, 19% adapted or adopted Level II MTS programs 
and practices. (1) 

F. Software Development Partnership Project 

• More than 60% of the revenues for the Software Development program were 
provided by software development organizations and partners in the project. (1) 

• Two of the four projects report that distribution of the product is meeting 
expectations, one reports that sales are much less than predicted, and the 
otherreported that the investment probably would not be recovered. (1) 

• Royalties from out-of-state sales from three of the products have already returned 
$105,000 to the state. 
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